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For  the  twentieth  year,  the  Research  and  Theory  Division  of  the  Association  for 
Educational  Communciations  and  Technology  (AECT)  is  sponsoring  the  publication 
of  these  Proceedings.  Papers  published  in  this  volume  were  presented  at  the  National 
AECT  Convention  in  St.  Louis,  MO.  A limited  quantity  of  this  volume  were  printed 
and  sold.  It  is  available  on  microfiche  through  the  Educational  Resources  Clearing- 
house (ERIC)  system. 

REFEREEING  PROCESS:  All  research  papers  selected  for  presentation  at  the 
AECT  convention  and  included  in  this  Proceedings  were  subjected  to  a rigorous  blind 
reviewing  process.  All  references  to  author  were  removed  from  proposals  before  they 
were  submitted  to  referees  for  review.  Approximately  fifty  percent  of  the  manuscripts 
submitted  for  consideration  were  selected  for  presentation  at  the  convention  and  for 
publication  in  these  Proceedings.  The  papers  contained  in  this  document  represent 
some  of  the  most  current  thinking  in  educational  communications  and  technology. 

A selected  number  of  development  papers,  sponsored  by  the  Division  for  In- 
structional Development  (DID),  are  included  in  this  Proceedings.  The  most  important 
instructional  development  papers  were  selected  by  the  DID  program  chairs  for  publi- 
cation. 

This  volume  is  indexed  by  both  author  and  descriptors.  The  index  for  volumes 
1-6  (1979-84)  is  included  in  the  1986  Proceedings,  and  the  index  for  volumes  7-10  is 
in  the  1988  Proceedings.  After  1988,  each  volume  contains  indexes  for  that  year  only. 
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AECT  has  eleven  divisions  to  meet  your  special  interests.  Each  division  publishes  its  own  newslelter  and  most  divisions  sponsor  an 
awards  program.  Your  AECT  membership  includes  one  division,  and  you  may  join  as  many  divisions  as  you  like  for  S 10  each. 


Division  for  Systemic  Change  In  Education  (CHANGE)  promotes 
systamic  change  In  schools,  businesses,  highar  education,  and  technical 
institutes,  primarily  Ihrough  systems  design  approaches,  '.omeet  learners’ 
needs,  improve  Ihe  quality  ol  education,  and  (or  enabling  technology  to 
reach  its  polential  in  all  educational  sellings. 

Olvislon  ol  Educational  Media  Management  (DEMM)  shares 
information  on  common  problems,  provides  solutions  and  program 
descriptions  ol  educational  media  management  programs  that  help  carry 
out  media  management  responsibilities  effectively. 

Division  ol  Instructional  Development  (DID)  studies,  evaluates,  and 
refines  design  processes  including  analysis  techniques  and  consultation 
strategies;  disseminates  research  findings  pertinent  to  these  processes: 
creates  new  models  ol  instructional  development:  and  promotes 
academic  programs  in  instructional  technology. 

Division  ol  Interactive  Systems  and  Computers  (DISC)  is  concerned 
with  ihe  generation,  access,  organization,  storage,  and  delivery  ol  all 
lorms  ol  Informalion  used  in  the  processes  ol  education  and  training.  The 
division  promotes  the  networking  ol  its  members  to  facilitate  sharing  oi 
expertise  and  interests. 

Division  ol  Learning  and  Performance  Environments  (DLPE) 
supports  human  learning  and  performance  Ihrough  Ihe  use  ol  computer- 
based  technology:  design,  development,  evaluation,  assessment,  and 
implementation  ol  learning  environments  and  performance  support 
systems  lor  adults. 

Division  of  Telecommunications  (OOT)  strives  to  improve  instruction 
Ihrough  the  use  of  telecommunications  including  television,  radio,  video, 
audio,  recordings,  teleconferences,  and  interactive  distance  education. 


DOT  is  concerned  wilh  Ihe  design,  production,  delivery,  utilization,  and 
evaluation  ol  instruclional  lolocommunications  malerials  and  equipment. 

Division  of  School  Media  Specialists  (DSMS)  enhances  communica- 
tions among  school  media  personnel  and  is  concerned  wilh  the 
development,  evaluanon,  and  implementation  of  school  media  programs 
designed  around  national  guidelines. 

International  Division  (INTL)  enhances  professional  and  persona! 
relationships  between  present  and  future  international  leaders  and 
encourages  practice  and  research  in  communication  lor  social  and 
economic  development  across  national  tines. 

Industrial  Training  and  Educalion  Division  ((TED)  Is  involved  wilh 
designing,  planning,  and  managing  training  programs;  and  works  lo 
promote  maximum  utilization  ol  educational  techniques  and  media  that 
are  effective  in  practical  use. 

Media  Design  and  Production  Division  (MDPO)  provides  an 
international  network  lhat  focuses  on  enhancing  the  quality  and 
effectiveness  of  mediated  communication,  in  all  media  lormals— <n 
educational,  governmental,  hospital,  and  corporate  sellings—lhrouqh  the 
interaction  ol  instructional  designers,  Irainers,  researchers,  and 
evaluators  with  media  designers  and  production  team  specialists  who 
utilize  slale  ol  the  art  production  skills. 

Research  and  Theory  Division  (RTD)  facilitates  the  design,  execution, 
utilization,  evaluation,  and  dissemination  ol  educational  teennofogy 
research;  promotes  the  applied  and. theoretical  research  on  the  use  of 
educational  lechnclcgy.  encourages  Ihe  use  of  multiple  research 
paradigms  in  examining  issues  related  lo  technology  in  instruction. 
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Association  for  Educational  Communications 
and  Technology 


What  is  AECT? 

AECT  is  the  only  national,  professional  association 
dedicated  to  the  improvement  of  instruction  through  the 
effective  use  of  media  and  technology.  AECT  assists 
its  members  in  using  technology  in  their  jobs  and  to 
enhance  the  learning  process. 

Who  belongs  to  AECT? 

□ Media  specialists 
Q Educators 

Q Librarians 

Q Instructional  designers 
Q Corporate/military  trainers 
Q Learning  resource  specialists 
Q Curriculum  developers 

□ Television  producers  and  directors 
Q Communications  specialists 

Q Education  administrators 
Q Others  who  require  expertise  in  instructional 
technology 

What  are  AECT  members  involved  In? 

Q Hypermedia 
Q Interactive  Video 
Q CD-ROM 

□ CDI 

Q Teleconferencing 
Q Film  and  Video  Utilization 
Q Telecommunications 

□ Computer  software  and  hardware 

□ Projection/presentation  products 
Q Intelligent  tutoring  systems 

□ Videodiscs 

□ Distance  learning 
Q And  morel 

AECTs  publications,  convention,  trade  show,  and 
conferences  present  the  leading  edge  on  research  and 
practical  applications  for  these  and  other  technologies. 

AECT  History 

Unlike  some  other  special-interest  technology 
organizations,  AECT  has  a long  history  with  over  70 
years  in  educational  technology.  We've  grown  up  with 
technology,  advocating  its  integration  into  education 
from  films  to  integration  into  education  from  films  to 
overheads  to  interactive  video  and  hypermedia. 
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AECT  began  as  the  Department  of  Visual  Instruction  at 
the  National  Education  Association  in  1923,  in  the  days 
when  visual  aids  consisted  of  films  and  slides.  In  1947, 
as  educators  were  adapting  technology  used  to  train 
World  War  II  service  personnel  for  the  classroom,  the 
name  of  the  organization  became  the  Department  of 
Visual  Instruction  (DAVI).  Twelve  years  later,  DAVI 
became  an  affiliate  of  the  NEA  and  finally  the 
autonomous  association,  AECT,  in  1 974. 

Today,  AECT  keeps  an  eye  on  the  future  of  instructional 
technology  while  assisting  educators  with  the  changes 
and  challenges  that  face  them  now.  AECT  members, 
now  numbering  4,500,  are  professionals  devoted  to 
quality  education.  They  care  about  doing  their  jobs 
better  and  want  to  embrace  new  methods,  new 
equipment,  and  new  techniques  that  assist  learning. 

Membership  in  AECT  increases  your  effectiveness,  your 
expertise,  and  your  skills.  These  qualities  in  turn 
enhance  your  professional  image  and  earning  potential. 

AECT  Affiliates 

AECT  has  47  state  and  16  regional  and  national 
affiliates,  and  has  recently  established  several  chapters 
surrounding  major  universities  and  metropolitan  areas. 
These  affiliated  organizations  add  a localized  dimension 
to  your  AECT  membership  and  allow  for  more 
interaction  among  your  colleagues.  For  more  details 
on  chapters  and  affiliates  in  your  area,  contact  the 
AECT  National  Office. 

AECT  National  Convention  and  InCITE 
Exposition 

Each  year,  AECT  brings  top  speakers  to  exciting 
locations,  and  presents  over  300  sessions  and  special 
events  to  provide  the  best  training  available  in  the  use 
of  media  in  education  and  instruction.  The  convention 
features  the  InCITE  Exposition,  the  first  trade  show 
created  exclusively  for  instructional  technology 
products.  At  InCITE,  you’ll  see  computers,  learning 
systems,  software,  interactive  multimedia,  audiovisual 
products,  films  and  videotapes,  projectors  and 
presentation  products,  video  equipment,  accessories, 
and  more.  The  convention  offers  tracks  of  sessions 
focusing  on  specific  interest  areas  surrounding  AECT’s 
nine  divisions  and  other  special  in4  rests.  The 
convention  has  featured  a Hypermedia  Strand  and  a 
Total  Quality  Management  Track.  In  addition,  intensive 
full  and  half-day  workshops  are  offered  for  in-depth 
training  on  the  latest  technology  applications  for 
education. 

Research  and  Theory  Division  (RTD)  improves 
the  design,  execution,  utilization,  evaluation,  and 
dissemination  of  educational  technology  research  and 
theory;  advises  educators  on  using  research  results. 
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An  Instructional  Model  for  Teaching  Literacy:  Implications  for  Instructional 

Theory 


Charoula  Angeli 
Indiana  University 


Abstract 

The  purpose  of  this  research  was  the  development  of  a learner-centered  instructional  model  for  the 
teaching  of  reading  and  writing  skills.  Specifically,  the  14  AHA  learner-centered  principles  have  been  used  as  the 
conceptual  framework  for  the  development  of  the  model.  There  are  seven  iterative  instructional  events  that 
comprise  the  instructional  model , and  they  are:  (I)  Establish  rapport  with  the  learner;  (2)  Engage  in  informal 
conversation;  (3)  Make  the  transition  from  informal  conversation  to  structured  dialogue;  (4)  Promote  awareness  of 
correct  spelling  and  proper  sentence  structure;  ( 5 ) Facilitate  the  construction  of  symbolic  knowledge;  (6)  Promote 
reflection;  and  (7)  Encourage  the  learner  to  assume  higher  agency  in  his  or  her  learning.  The  model  has  been 
developed  using  qualitative  methods  of  research  and  50  elementary  school  children  participated  in  the  study . 
Lastly,  this  study  promotes  a learner-centered  paradigm  of  instruction,  one  that  is  fundamentally  different  from 
traditional  practices,  as  it  places  the  learner  in  the  center  of  the  learning  process  and  emphasizes  meeting 
individual  learning  needs  at  different  rates. 

Introduction 

There  is  no  doubt  that  one  of  the  most  controversial  issues  in  education  for  several  decades  has  been  the 
teaching  of  literacy  (Noll,  1995).  The  debate  has  often  centered  on  two  different  approaches  of  teaching  reading  and 
writing,  namely  the  phonics  approach  and  the  whole  language  approach.  Advocates  of  the  phonics  approach  (e.g.. 
Chall,  1967;  1989)  argue  that  the  whole  language  approach  is  not  an  effective  method  for  leaching  literacy,  because 
it  does  not  provide  enough  structure.  On  the  other  side  of  the  debate,  Goodman  (1986),  an  advocate  of  the  whole 
language  approach,  asserts  that  the  phonies  approach  makes  language  learning  difficult,  because  it  docs  not  lake  into 
consideration  the  needs  and  experiences  of  learners.  No  matter  which  side  of  the  debate  one  belongs  to,  the  fact  that 
there  are  currently  14  million  children  in  the  United  States  who  have  failed  to  reach  satisfactory  levels  of  literacy 
cries  for  our  immediate  attention,  fresh  conceptions  of  literacy,  and  innovative  ways  to  support  its  development. 

One  promising  approach  has  been  to  focus  on  the  importance  of  dialogue  in  literacy.  Vygotsky  (1962)  has 
proposed  that  learning  is  a social  process,  and  that  children  first  learn  “social  speech,’'  talk  directed  at  others,  in  their 
attempt  to  communicate  with  their  significant  others.  He  argues  that  children’s  social  speech  is  later  turned  inwards, 
and  experienced  as  thought.  Oakshott  (1962)  draws  attention  to  the  critical  relationship  between  language,  thought, 
and  dialogue.  He  hypothesizes  that  humans  have  evolved  a dialogic  competence  that  is  both  private  and  public. 
This  dialogue  can  lake  place  in  collaboration  with  other  people  in  a social  environment,  or  it  can  take  place  within 
ourselves  in  the  form  of  an  internal  dialogue,  often  called  reflective  thought. 

Tharp  and  Gallimore  (1988)  assert  that  literacy  events  should  be  experienced  by  learners  as  collaborative 
social  activities  with  goals  embedded  in  natural  settings,  and  not  as  isolated  and  dccontextuali/cd  events.  Along  the 
same  line  of  reasoning,  Bruffcc  (1984),  who  views  dialogue  as  a vital  element  in  composing  thoughts,  claims  that 
collaborative  writing  groups  provide  the  social  context  and  the  structure  for  such  dialogue  to  take  place.  In  arguing 
that  al!  knowledge  is  constructed  and  that  social  interactions  are  critical  in  knowledge  construction,  he  lays  the 
theoretical  groundwork  for  using  dialogues  and  conversations  among  socially  interacting  people  jointly  negotiating 
text.  Hall  (1987)  also  agrees  that  loaners  develop  acceptable  literacy  skills  when  they  arc  involved  in  meaningful 
and  collaborative  activities. 

As  Downing  (1979)  asserts,  however,  literacy  involves  a mark-making  and  interpretive  process  that 
requires  explicit  awareness  of  the  meta-linguistic  aspects  of  language.  Awareness  of  those  aspects  of  language 
required  for  literacy  docs  not  develop  spontaneously  from  spoken  language;  they  arc  inaccessible  to  one’s 
consciousness  without  the  aid  of  a teacher,  mentor,  or  collaborative  community.  Hence,  what  is  required  for  literacy 
is  a language  awareness  support  structure,  which  incorporates  the  more  conscious,  deliberate,  meta-linguistic 
processes  that  help  make  meaning  in  reading  and  writing. 
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Bubble  Dialogue,  a computer  software  tool  developed  by  the  Language  Development  and  Hypermedia 
Research  Group  (Cunningham  ct  aL  1992),  was  designed  to  provide  such  a language  awareness  support  structure 
for  the  acquisition  of  literacy.  In  this  study,  50  elementary  school  children  used  Bubble  Dialogue  as  a tool  to  help 
them  improve  their  reading  and  writing  skills.  Before  I proceed,  I would  like  to  first  introduce  Bubble  Dialogue  and 
discuss  briefly  the  language  awareness  structure  it  supports. 

Bubble  Dialogue 

Bubble  Dialogue  is  a HyperCard  application  that  combines  elements  of  role  play,  comic  strip  creation,  and 
reflexive  dialogue  analysis  (The  Language  Development  and  Hypermedia  Research  Group,  1992).  A master  stack, 
called  BubblcMaker  (see  Figure  1)  has  been  written  to  create  customi/.cd  Bubble  Dialogue  slacks  for  later  use.  The 
scene  in  which  the  discussion  is  to  lake  place  is  established  by  the  participants.  A graphic  is  then  chosen,  and  the 
setting  is  described  by  writing  a prologue.  In  each  customized  stack  created  by  BubblcMaker.  four  icons, 
representing  a speech  bubble  (public/social  speech)  and  a think  bubble  (inner  speech)  per  character,  are  presented 
alongside  at  least  two  characters  on  the  screen.  The  comic  genre  is  so  well  established  in  many  cultures  that  even 
very  young  children,  when  presented  with  empty  bubbles,  feel  compelled  to  speak  for  the  characters  playing  out 
their  roles. 


Figure  /.  The  Bubble  Maker  workspace 

Bubble  Dialogue  can  operate  in  two  distinct  modes;  the  creation  mode  and  the  review  mode.  In  the 
creation  mode  (see  Figure  2),  one  can  only  move  forward  to  the  next  empty  think  or  speech  bubble.  When  in  this 
generative  mode,  the  course  of  action  is  transferred  to  the  other  character  once  a speech,  and  optionally,  one  think 
bubble  have  been  used.  This  turn-taking  protocol  ensures  that  a user  cannot  have  an  extended  conversation  with 
himself  or  herself. 
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Figure  2.  The  tool  in  creation  mode 

In  contrast,  in  review  mode  (sec  Figure  3).  the  user  can  move  forward  or  backward  to  add  notes,  edit 
existing  bubbles,  or  even  extend  the  dialogue.  Most  importantly,  the  review  mode  provides  the  user  with  an 
opportunity  to  reflect  on  the  dialogue  that  took  place,  as  well  as  the  quality  of  his  or  her  writing.  One  of  the  features 
that  is  activated  while  the  application  is  in  review  mode  is  the  notes  field.  This  feature  is  powerful  in  a variety  of 
ways.  For  example,  when  users  review  their  dialogues  they  can  use  the  notes  field  to  add  comments  about  "what's 
happening.’*  or  comments  on  the  motives  and  feelings  of  the  characters.  Teachers  and  researchers  may  also  use  this 
feature  to  annotate  the  dialogues  in  any  manner  they  sec  fit,  such  as  commentary,  reminders,  and  questions  for  the 
users  to  consider  as  they  review  their  work. 

All  these  features  of  Bubble  Dialogue  make  up  the  language  awareness  support  structure  that  is  needed  for 
learners  to  make  the  transition  from  conversation  to  written  dialectic  discourse.  Think  and  speech  bubbles  enable 
(he  learner  to  function  in  both  (he  interpersonal  and  intrapcrsonal  levels  {Vygotsky,  1962).  as  well  as  learn  to  take 
turns  in  a dialogue.  In  addition,  the  review  mode  allows  learners  to  reflect  and  revise  their  dialogues  appropriately. 


Research  Participants 

Bubble  Dialogue  was  used  in  collaboration  with  school  children  to  write  stories.  Fifty  elementary  school 
children  in  a midwestern  city  of  about  60.000  used  Bubble  Dialogue  over  a period  of  seven  months.  Thirty  of  those 
students  were  in  second  grade,  five  were  in  third  grade,  and  fifteen  were  students  from  a multiagc  classroom  for 
fourth,  fifth,  and  sixth  grade  levels.  Fifteen  of  the  fifty  students  were  reported  as  students  with  special  needs,  and 
five  were  identified  as  gifted.  Thirty-eight  of  the  students  were  Caucasian,  ten  were  African-American,  one 
Hispanic-Amcrican,  and  one  Amcrican-Indian. 

Bubble  Dialogue  sessions  lasted  for  about  45  minutes  to  60  minutes  each.  Younger  children  (i.c.,  first, 
second,  and  third  graders)  needed  more  lime  to  get  used  to  the  computer  tool  and  to  type  their  responses.  Each  story 
(he  children  wrote  was  saved  in  a file  by  Bubble  Dialogue  and  was  primed  out  at  the  end  of  each  session. 
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Figure  3.  The  tool  in  review  mode 


Theoretical  Framework 

The  instructional  model  that  is  developed  is  based  on  lcarncr-ccntcrcd  principles  for  learning  and 
instruction.  The  14  learner-centered  psychological  principles  (LCPs)  provide  a good  framework  for  curriculum 
design  and  instruction  (American  Psychological  Association,  1995).  The  principles  arc  divided  into  four  categories: 
(1)  Cognitive  and  mctacognitivc;  (2)  Motivational  and  affective;  (3)  Developmental  and  social;  and  (4)  Individual 
differences. 

In  brief,  the  LCPs  state  that  learning  occurs  more  effectively  when  it  is  an  intentional  process  of 
constructing  meaning  from  experience  and  information.  This  view  emphasizes  that  learners  need  the  support  and 
instructional  guidance  of  a mentor,  coach,  or  collaborative  community  to  learn  how  to  create  meaningful 
representations  of  knowledge  and  engage  in  projects  that  will  help  them  pursue  their  goals.  In  addition,  learners 
need  assistance  in  constructing  and  integrating  new  knowledge  with  existing  knowledge,  in  part,  by  using  a variety 
of  cognitive  strategics.  Salient  in  the  14  LCPs  is  the  development  of  the  learner’s  thinking  processes  and  strategics. 
Successful  learners  arc  the  ones  who  arc  engaged  in  problem-solving  and  reflection,  have  developed  reasoning 
skills,  and  know  how  to  apply  prior  knowledge  to  new  situations.  But  the  LCPs  also  emphasize  the  importance  of 
motivation.  Curiosity,  perceived  relevance  of  the  task  to  the  learner’s  goals,  and  personal  choice  and  control,  for 
instance,  arc  all  factors  that  contribute  to  a learner’s  motivation.  Learning  is  also  influenced  by  the  learner’s 
developmental  level,  as  well  as  by  how  he  or  she  interacts  in  social  environments  when  collaborating  with  others. 

Instructional  Model 

I would  like  now  to  discuss  the  instructional  model  that  is  modeled  after  the  lcarncr-ccntcrcd  principles 
for  learning  and  instruction  Based  on  the  design  characteristics  of  Bubble  Dialogue  and  the  theoretical  framework  of 
LCPs,  I developed  the  instructional  model  shown  in  Figure  4 below. 

In  accordance  with  the  LCPs,  learning  and  instruction  were  viewed  as  cognitive,  mctacognitivc, 
developmental,  and  social  processes.  Instruction,  therefore,  took  on  many  different  dimensional  forms  but  always 
within  the  framework  of  dialectic  discourse. 

The  instructional  process,  shown  in  Figure  4 above,  consisted  of  seven  different  phases. 
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Phase  I 

First  of  all,  the  researcher  established  rapport  with  the  students.  The  goat  during  this  first  stage  was  to  gel 
to  know  the  students  better  and  inform  them  of  the  purpose  of  the  computer  activity.  Even  though  it  was  desired  to 
make  the  instructional  process  purposive  and  intentional,  the  researcher  also  wanted  the  students  to  find  the  task 
relevant  to  their  interests.  As  Short  and  Burke  (1991)  slate,  '‘whenever  students  are  involved  in  experiences  where 
they  have  choices,  there  is  a greater  likelihood  that  they  will  be  able  to  make  choices  that  allow  them  to  connect  with 
what  they  already  know"  (pg.  36).  Therefore,  students  were  asked  to  draw  from  their  life  experiences  and  select  the 
topic  they  wanted  to  write  a story  about.  Older  students  (in  third,  fourth,  fifth,  and  sixth  grade  levels)  easily  referred 
to  their  family  life  or  hobbies  to  select  the  topic  they  were  interested  in  talking  about,  but  when  the  younger  second* 
graders  were  given  the  option  to  choose,  they  claimed  that  they  did  not  know  what  they  wanted  to  talk  about. 
Therefore,  this  introductory  phase  lasted  longer  with  second-graders,  as  they  needed  more  help  in  selecting  a topic 
that  was  of  interest  to  them. 

Phase  II 

After  the  researcher  and  the  students  established  a topic,  they  engaged  in  purposive  conversation  to  learn 
more  about  it.  Though  this  phase  did  not  last  long,  it  was  used  as  a bridge  to  shift  from  casual  conversation  to 
conscious  dialogue.  Initially,  all  children  found  it  difficult  to  gel  familiar  with  the  structure  of  dialectic  discourse 
that  Bubble  Dialogue  provided.  The  students  had  to  be  constantly  reminded  that  they  had  to  use  the  computer  to 
type  their  responses  in  cither  a speech  or  a think  bubble. 

Phase  111 

Once  the  transition  from  conversation  to  dialogue  was  modeled,  the  researcher  modeled  using  the  speech 
and  think  bubbles  for  the  students,  and  asked  them  if  they  knew'  the  difference  between  the  two.  Since  many 
students  initially  fell  hesitant  and  unsure  about  the  use  of  think  bubbles,  they  avoided  them.  The  researcher 
continued  modeling  their  uses  for  the  students,  and  noticed  that  after  four  or  five  speech  bubbles,  they  tried  at  least 
one  think  bubble. 
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Phase  IV 

Once  the  students  felt  comfortable  v.  all  Bubble  Dialogue,  the  researcher  asked  some  of  them  to  read  aloud 
what  they  were  typing  in  the  bubbles.  The  reason  for  this  request  was  to  increase  their  metacognitivc  awareness 
about  the  mechanics  of  their  language  use.  since  many  of  them  were  misspelling  words  and  writing  sentences  that 
were  not  grammatically  correct.  Once  they  began  reading  aloud  what  they  wrote,  they  became  aware  of  the  fact 
that  what  they  wrote  did  not  “sound”  too  well.  Accordingly,  they  began  asking  questions  about  how  to  spell  certain 
words,  if  they  used  punctuation  correctly,  and  if  their  sentences  were  well-formed. 

Postman  (1995)  states  that  “there  arc  many  ways  to  teach  the  young  the  connections  between  language  and 
world-making”  (pg.  84),  and  he  argues  that  teachers  make  a major  mistake  by  focusing  students’  language  use  on 
how  to  spell  words  correctly  and  use  punctuation  rules  properly.  Therefore,  literacy  becomes  a game  of  trivia.  As  a 
result,  during  the  fourth  phase,  the  role  of  the  researcher  as  the  “teacher"  became  extremely  difficult  and 
challenging.  As  Sternberg  and  Horvath  (1995)  note,  a good  teacher  is  one  that  is  insightful  and  one  that  holds  a bag 
of  different  tools  to  use  appropriately.  Since  not  all  students  were  at  the  same  level  of  language  development,  the 
biggest  challenge  for  the  researcher  was  lo  diagnose  when  the  time  was  right  to  make  a student  aware  about  the 
technical  aspects  of  language.  Some  (10%)  students  were  aware  from  the  beginning  about  their  language  use  by  just 
watching  the  way  the  researcher  wrote.  More  (60%)  gradually  became  aware  of  their  writing  as  the  researcher  was 
constantly  asking  them  questions  to  think  about  their  writing.  The  remaining  students  (30%)  failed  to  discriminate 
between  the  researcher’s  writing  and  their  writing. 

Cunningham  (1992)  stales  that  knowledge  docs  not  consist  of  a number  of  objects  that  learners  acquire, 
instead  knowledge  is  better  thought  of  as  knowing  or  as  a process.  In  tandem  with  this  notion,  literacy  was  viewed 
as  a process  of  how  learners  conic  to  mature  thought  and  reason  about  abstract  concepts  (Bruner,  1966)  and  world- 
views  (Postman,  1995).  At  the  same  time,  it  was  recognized  that  it  was  important  to  learn  how  lo  write  correctly. 
The  strategy  for  dealing  effectively  with  a number  of  different  zones  of  proximal  development  (Vygotsky,  1962), 
therefore,  was  to  first  assess  the  readiness  of  students  for  new  learning,  and  then,  through  a process  of  appropriation 
(Rogoff,  1990),  introduce  them  to  grammatical  and  punctuation  rules  by  using  language  lo  talk  about  language. 
Therefore,  not  all  children  entered  the  fourth  phase  at  the  same  pace.  For  those  students  (10%)  who  were  aware 
about  their  language  use  from  the  beginning,  the  instructional  process  shown  in  Figure  4 above  was  systematically 
used  in  a linear  fashion.  In  contrast,  students  who  gradually  became  aware  of  their  writing  (60%),  were  guided  from 
phase  three  to  five  and  returned  to  four  at  a later  time.  There  were  two  reasons  for  this  tactic;  (1)  the  researcher 
wanted  the  students  lo  question  by  themselves  the  quality  of  their  writing,  and  ask  for  help  when  they  felt  they 
needed  it,  and  (2)  the  researcher  did  not  want  students  lo  think  that  the  only  goal  of  the  activity  was  lo  learn 
punctuation  and  grammatical  rules.  The  rest  of  them  (30%)  were  also  guided  from  phase  three  to  five  and  returned 
lo  four  when  the  researcher  fell  that  it  was  appropriate  to  introduce  language  mechanics. 

Phase  V 

During  the  fifth  phase,  the  goal  was  to  help  children  develop  their  cognitive  skills  by  asking  them  questions 
about  the  topic  they  selected.  Initially,  the  researcher’s  questions  were  simple  and  required  them  to  recall  certain 
facts.  Gradually,  she  shifted  lo  questions  seeking  their  personal  experiences  to  reason  and  make  inferences  about  the 
topic.  For  example,  if  the  topic  was  friends,  she  asked  questions  such  as:  “Why  do  you  like  Tim  as  a friend?’’, 
“What  qualities  should  a friend  have?”,  “Arc  friends  important?”,  “What  do  you  do  to  be  a good  friend  to 
somebody?”.  The  goal  was  to  activate  their  belief  system  about  friendship  through  a process  of  inductive  and 
deductive  questioning.  When  the  question  was  one  that  they  had  not  encountered  before,  the  researcher  helped  them 
lo  build  a context  within  which  the  question  could  be  answered  meaningfully. 

This  last  process  is  known  as  abduction  (Cunningham  1992;  Shank,  1987).  The  instructional  strategy  used 
for  fostering  abduction  was  to  present  students  with  a perspective  different  than  their  own;  most  of  the  time  this  was 
the  researcher’s  own  perspective.  Therefore,  students  came  to  know  how  their  knowledge  templates  and  belief 
system  were  situated  in  relation  to  somebody  clsc’s  belief  system,  while  enhancing  theirs  as  needed. 


Phase  V 


The  sixth  phase  of  reflection  and  reflcxivity1  was  at>o  of  particular  interest.  Promoting  reflection  and 
rcflcxi viiy  was  an  instructional  strategy  that  was  used  primarily  to  help  students  develop  mclacognitivc  and  higher- 
order  thinking  skills.  The  terms  reflection  and  rcllcxivity  arc  defined  as  “reflection  of  our  reflections,  thinking  about 
our  thinking  process,  knowing  how  we  know”  (Cunningham.  1992,  p.  1 87).  The  researcher’s  goal  was  to  encourage 
students  to  think  about  the  dialectic  discourse  reported  in  speech  bubbles,  analyze  the  arguments  presented, 
synthesize  all  that  was  said  so  far,  edit  and  revise  appropriately,  and  decide  how  to  respond  next. 

The  findings  indicate  that  students  found  it  cither  difficult  or  irrelevant  to  use  think  bubbles  as  they  made 
limited  use  of  them.  Nonetheless,  when  they  did  employ  them,  they  used  them  to  evaluate  their  current  state  of 
knowledge  and  say  “I  don’t  know,  why  is  she  asking  me  that  question?”,  or  express  a concern  they  had  about  what 
the  other  person  had  said.  When  the  researcher  asked  students  why  they  were  not  using  think  bubbles  as  much  as 
intended,  they  responded  that  they  did  not  have  to  think  about  what  to  say,  they  already  knew.  However,  as  it  was 
observed,  students  stopped  and  thought  on  numerous  occasions  about  their  answer  without  using  a think  bubble. 
Therefore,  it  seems  that  students  either  felt  that  it  was  too  much  work  to  type  in  their  thoughts,  or  they  did  not  sec 
the  connection  between  speech  and  think  bubbles  and  how  they  could  use  both  to  shape  their  internal  and  external 
dialogue.  It  is  the  researcher’s  belief  that  a certain  amount  of  practice  with  Bubble  Dialogue  is  needed  for  a student 
to  make  the  connection  between  think  and  speech  bubbles.  It  was  also  expected  that  the  notes  field  feature  of 
Bubble  Dialogue  would  provide  an  appropriate  opportunity  for  reflcxivity  to  take  place,  but  students  also  made 
limited  use  of  this  feature.  Here,  again,  it  is  proposed  that  with  increased  use  of  Bubble  Dialogue,  reflexive  use  of 
the  notes  field  and  other  features  will  increase. 

Phase  VII 

In  all  six  stages  of  the  instructional  process  used  here,  the  "teacher”  provided  scaffolding  while  the  student 
was  constructing  a set  of  negotiated  beliefs.  But  even  in  this  prevailing  teaching  model  it  was  the  teacher  who  asked 
questions,  directed  inquiry,  and  provided  direction.  As  a result,  the  learners  were  dependent  on  the  teacher's 
assessment  of  what  constitutes  knowledge,  because  the  teacher  was  the  one  who  was  in  control  of  movements  within 
the  zone  of  proximal  development  (ZPD).  To  help  them  grow  out  of  this  dependency,  the  researcher  tried  another 
activity  using  Bubble  Dialogue  which  encouraged  learners  to  assume  higher  levels  of  agency  in  their  learning 
process  by  asking  the  questions,  thus  directing  the  dialogue.  This  movement  out  of  dependency  is  really  a question 
of  ZPD  control.  Therefore,  in  attempting  to  shift  control  of  the  ZPD  from  the  teacher’s  control  to  mutual  control, 
the  researcher  hoped  to  balance  the  learning  process.  Previous  research  (e.g..  Brown  & Pal  inscar.  1989;  Palinscar, 
1986;  Scardamalia  & Bcrcilcr,  1985)  has  shown  that  such  a shift  in  power  and  control  can  be  achieved  only  in 
learning  environments  that  support  the  dialectic  process,  wherein  learners  with  the  appropriate  scaffolding  lake  over 
the  asking  of  questions,  and,  thereby,  direct  their  own  inquiry.  Bubble  Dialogue  proved  to  be  an  effective  vehicle 
for  supporting  the  development  of  such  student  self  agency.  Student  opportunities  for  self  agency  arc  evident  in  the 
analyses  of  this  study’s  dialectic  discourse. 

Implications  for  Instructional  Theory 

I would  like  now  to  discuss  the  implications  of  this  research  study  for  instructional  theory  and  practice. 
Rcigclulh  (in  press)  advocates  a new  paradigm  of  instructional  theory,  a learner-focused  paradigm  as  he  calls  it.  The 
term  instructional  theory  here  is  used  the  same  way  as  Rcigclulh  uses  it,  that  is  instructional  theory  is  a design  theory 
that  offers  guidance  on  how  to  facilitate  people’s  learning.  Traditionally,  instructional  practices  emphasized  the 
fulfillment  of  learning  goals,  always  prc-dctcnnincd  by  the  teacher,  at  specific  rales.  In  other  words,  all  learners 
were  expected  to  reach  the  same  learning  goals  at  the  same  amount  of  lime.  Therefore,  teaching  has  been  viewed  as 
a mechanical  process  of  disseminating  information  to  learners;  a one-way  interaction  originating  from  the  teacher  to 
a group  of  learners. 

What  really  the  findings  of  this  study  show  is  that  teaching  and  learning  cannot  be  viewed  as  one-way 
interactions,  but  instead  as  collaborative  processes  situated  in  real,  authentic,  and  social  contexts  in  which  the  goal  is 
to  meet  each  learner’s  individual  needs.  As  it  was  discussed  earlier,  not  all  learners  in  this  study  were  able  to 
complete  all  instructional  events  at  the  same  time.  As  it  was  indicated,  instructional  lime  varied  from  45  to  60 
minutes.  Therefore,  instructional  theory  should  be  adaptive  in  nature.  What  I mean  by  adaptive  is  ( 1 ) to  be  able  to 


1 Cunningham  ( 1992)  differentiates  reflcxivity  from  reflection,  but  in  this  paper  there  is  no  distinction  between  the 
two. 
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customize  instruction  accordingly  to  adhere  to  the  needs  of  each  learner,  and  (2)  to  allow  learners  to  attain 
competencies  at  different  rales. 

Another  finding  from  this  research  indicates  that  learners  should  he  encouraged  to  assume  higher  levels  of 
agency  in  their  learning.  Traditionally,  the  teacher  has  been  the  primary  agent  that  controlled  the  zone  of  proximal 
development  of  each  learner  by  dictating  what  needs  to  he  learned,  when,  and  in  what  sequence.  The  role  of  the 
teacher  in  this  new  paradigm  of  instructional  design  theory  is  still  of  paramount  importance,  hut  different.  In  this 
study,  the  teacher  and  the  learner  worked  together  to  design  an  instructional  episode  with  Bubble  Dialogue  (to 
remind  the  reader,  the  first  event  in  the  model  is  the  establishment  of  rapport  with  the  student  in  order  to  decide  what 
story  the  student  wants  to  write  using  Bubble  Dialogue).  It  is  important  to  have  shared  control  in  this  Icarncr- 
focuscd  paradigm.  The  student  should  be  asked  to  make  certain  decisions,  thus  allowing  him  or  her  to  direct  the 
learning  process  in  a way  that  is  relevant  and  motivating  to  him  or  her,  not  relevant  or  niotivaiing  to  'he  teacher. 

Lastly,  I would  like  to  stress  the  importance  of  designing  instruction  that  is  situated  in  a real  activity  that 
makes  sense  to  the  learner.  Within  the  context  of  authentic  activities  learning  new  information  becomes  meaningful 
and  relevant.  Therefore,  knowledge  docs  not  remain  inert  (Whitehead,  1957),  instead  it  becomes  usable.  In  this 
research  study,  learners  learned  about  spelling  and  how  to  construct  well-formed  sentences  within  the  context  of 
writing  a story  as  a collaborative  activity  between  them  and  the  teacher.  In  other  words,  the  teaching  of  literacy  was 
not  approached  as  an  act  of  disseminating  decontcxtualized  knowledge  to  the  learners.  When  the  teachers  of  the 
school  children  read  the  stories  the  children  wrote,  they  were  amazed  by  how  different  the  children’s  written  speech 
appeared  from  the  one  in  their  journals.  Specifically,  they  noticed  that  in  genera!  students  wrote  more  than  what 
they  usually  wrote  in  their  journals,  formed  better  sentences,  and  made  less  spelling  mistakes. 

Conclusion 

In  this  paper  1 discussed  the  development  of  a learner-centered  instructional  model  for  the  teaching  of 
literacy.  The  instructional  model  is  based  on  the  theoretical  framework  of  the  14  APA  learner-centered  principles 
for  learning  and  instruction,  and  it  consists  of  seven  distinct  instructional  phases  or  events.  They  are:  ( 1 ) 
Establishment  of  rapport  with  learners;  (2)  Engagement  in  informal  conversation;  (3)  Transition  from  informal 
conversation  to  dialogue;  (4)  Awareness  of  spelling  and  sentence  structure;  (5)  Construction  of  symbolic 
knowledge;  (6)  Reflection;  and  (7)  Promotion  of  learners  as  higher  agents  in  the  learning  process.  1 then  discussed 
how  this  instructional  model  calls  for  a new  paradigm  of  instructional  theory;  one  that  places  the  learner  in  the 
center  of  the  learning  process,  and  one  that  breaks  away  from  the  notion  that  learning  requirements  should  be 
fulfilled  by  all  learners  at  the  same  time.  Finally,  I emphasized  the  importance  of  designing  instruction  that  is 
relevant  and  meaningful  to  learners. 
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As  wc  begin  the  second  decade  of  the  development  of  hypermedia  applications  for  personal  computers,  it 
seems  appropriate  to  examine  perhaps  the  most  critical  issue  in  the  design  of  large  hypermedia  databases,  namely 
navigation.  Applications  designed  today  look  and  feci  quite  different  from  those  created  over  ten  years  ago.  The 
difference  is  more  than  the  addition  of  color.  Today’s  applications  employ  different  strategies  or  philosophies  for 
navigation.  As  the  navigation  metaphor  suggests,  learners  must  move  about  in  a database  in  order  to  access  the 
information  stored  in  it.  With  a small  view  screen,  effective  navigation  is  oi  paramount  importance  for  complex  and 
possibly  vast  hypcrspaces.  Clearly  the  method  by  which  a user  selects  relevant  information  and  instruction  to  appear 
on  the  screen  in  this  computer  environment  will  have  a great  deal  to  do  with  effectiveness  of  the  resulting 
instruction. 

Although  theorists  in  such  Holds  as  educational  technology,  computer  science,  psychology,  and  education 
have  written  extensively  about  the  components  of  good  instructional  design,  the  application  of  these  theories  to 
hypermedia  has  not  always  been  satisfactory.  Compounding  the  problem  is  the  fact  that  many  of  the  proposed 
theories  conflict.  Research  into  the  effectiveness  of  various  hypermedia  designs  is  still  in  its  youth.  Consequently, 
there  arc  currently  no  convenient  foolproof  rules  to  guide  hypermedia  designers.  This  paper  will  examine  the  main 
issues  in  hypermedia  navigation,  propose  research  questions,  and  suggest  a methodology  for  resolving  those 
questions. 

What  Is  Navigation? 

Navigation  implies  controlled  movement  from  place  to  place.  The  American  Heritage  Dictionary  (2nd  cd., 
1982)  defines  navigation  to  be  “the  theory  and  practice  of  navigating,  especially  the  charting  of  a course  for  a ship  or 
aircraft”  (p.  833).  Newer  electronic  publications  do  not  significantly  extend  that  older  definition  even  though  they 
themselves  refer  to  “navigation”  within  their  own  hypermedia  applications.  For  example,  the  Grolier  Multimedia 
Encyclopedia  (Version  8.0.3,  1996)  defined  navigation  to  he  “the  art  and  science  of  conducting  a ship,  airplane,  or 
spacecraft  safely  and  expeditiously  to  a specific  dcstinatL  ..  In  the  broadest  sense  anyone  faced  with  the  problem  of 
‘finding  the  way’  is  a navigator,  whether  he  or  she  travels  on  land,  at  sea,  in  the  air,  or  in  space"  (Art  1 239). 

As  Conklin  (1987)  pointed  out,  the  word  “navigation,”  when  applied  to  hypermedia  applications,  lakes  on  a 
distinctly  different  meaning.  In  the  world  inside  a computer,  the  traveler  no  longer  need  be  on  a ship  nr  plane  and  to 
physically  chart  a course  of  travel  in  the  sense  of  the  dictionary  definition.  The  concept  of  navigation  in  hypermedia 
refers  to  figurative  movement  within  a potentially  complex  database  of  links  and  nodes,  rather  than  physical 
movement. 

The  term  navigation  has  become  a metaphor  for  motion.  Gygi  (1990)  stated  that  “navigation  is  a metaphor 
used  to  impart  some  sense  of  intcnlionality  as  well  as  spaliality”  (p.  281).  She  pointed  out  that  in  hypermedia 
applications  information  is  not  brought  to  the  user;  rather  the  user  must  visit  different  places  in  the  database  to 
access  the  information.  Scllcn  and  Nicol  (1990)  reported  that  users  make  heavy  use  of  spatial  metaphors  such  as 
saying  they  are  lost,  they  have  reached  a dead-end,  they  are  going  up  and  down  to  different  levels,  and  they  are 
moving  forward  and  backward.  Scllcn  and  Nicol  (1990)  contended  that  users  often  construct  spatial  mental  models 
to  enable  them  to  deal  with  “moving"  from  one  context  to  another  witlvu  a computer  application.  Such  mental 
models  arc  helpful  for  users  to  navigate  even  though  the  models  do  not  in  any  way  represent  the  actual  physical 
location  of  information  within  the  database  itself. 

Gygi  (1990)  and  McAleese  (1989)  drew  a distinction  between  browsing  and  navigation.  They  explained 
that  browsing  refers  to  users'  discovering  information  by  traversing  the  entire  database  in  an  unconstrained,  even 
random,  manner,  whereas  navigation  refers  to  users'  searching  for  information  in  a purposeful  way  in  portions  of  the 
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database  where  they  can  reasonably  expect  to  find  what  they  want  to  know.  Canter,  Rivers,  and  Storrs  (1985) 
developed  five  categories  for  user  navigation:  scanning  means  covering  a large  area  without  much  depth;  browsing 
is  following  any  path  until  an  area  of  interest  appears;  searching  means  striving  to  find  an  explicit  target;  exploring 
is  finding  out  the  extent  and  nature  of  the  database;  and  wandering  signifies  purposeless  and  erratic  travel  with 
recursion  (revisiting  nodes).  McAleese  (1989)  and  Horncy  (1993)  attempted  to  characterize  these  different  types  of 
navigation  by  studying  the  navigational  patterns  of  users.  All  five  of  these  categories  may  be  considered  to  he 
subsumed  under  the  general  heading  of  navigation. 

Design  Problems  in  Hypermedia  Databases 

Long  before  hypertext  was  a concept  hatched  in  Vanncvar  Bush's  fertile  mind  (Bush,  1945),  the  navigation 
of  information  loomed  large  as  a consideration  in  learning  environments.  When  such  instructional  materials  are 
stored  in  a computer,  designers  no  longer  have  the  luxury  of  physical  space  afforded  them  in  other  teaching 
environments  such  as  classrooms  with  extensive  chalkboards,  wails,  or  even  tables  to  spread  out  materials.  With 
computers,  all  exchanges  between  learners  and  instructional  materials  must  be  conducted  through  the  myopic  eye  of 
a computer  screen.  Educators  proficient  in  using  wide-angle-lens  techniques  listed  above  to  bring  the  world  to  their 
students  must  now  constrain  themselves  to  using  a technology  that  views  the  world  through  a relatively  restrained 
aperture. 

In  the  emerging  field  of  hypermedia  instruction,  becoming  lost  or  disoriented  is  potentially  the  most 
difficult  design  problem  to  solve.  The  number  of  theories  about  the  best  way  to  navigate  in  hypermedia  databases  far 
surpass  the  amount  of  research  that  has  been  performed  to  justify  those  theories.  Compounding  the  problem  is  the 
fact  that  many  of  the  proposed  theories  conflict.  The  current  cost  of  producing  professional  hypermedia  applications 
for  research  purposes  is  so  high  that  this  trend  is  likely  to  continue  for  some  time.  From  a user-centered  point  of 
view,  it  would  appear  that  hypermedia  applications  should  provide  users  with  an  entire  arsenal  of  navigational  tools, 
rather  than  force  them  to  rely  on  just  one  strategy  that  may  not  necessarily  suit  them. 

The  implementation  of  non-linear  access  to  many  forms  of  information  — text,  sound,  still  images,  video, 
animation,  and  the  like  — in  one  package  driven  by  a compuicr  is  a recent  development  in  education,  not  much 
more  than  ten  years  old.  Although  theorists  in  such  fields  as  educational  technology,  computer  science,  psychology, 
and  education  have  written  extensively  about  the  components  of  good  instructional  design,  the  application  of  these 
theories  to  hypermedia  has  not  always  been  satisfactory.  Research  into  the  effectiveness  of  various  hypermedia 
designs  is  still  in  its  infancy.  At  present,  there  arc  no  convenient  rules  to  guide  hypermedia  designers.  Consequently, 
the  design  of  hypermedia  applications  is  an  art  and  likely  will  remain  so  for  the  foreseeable  future. 

Perhaps  the  foremost  question  in  navigation  is  bow  to  avoid  becoming  “lost  in  hypcrspacc."  The  questions 
considered  below  attempt  to  examine  the  potential  causes  of  and  cures  for  disorientation.  A second  idea  for 
consideration  is  whether  or  not  “hypermaps"  should  be  provided.  In  a hypermedia  application,  not  every  piece  of 
information  needs  to  be  explicitly  linked  every  other  piece  of  information  and,  lienee,  a third  area  of  interest  is  the 
balance  between  links  and  nodes.  How  many  links  arc  necessary  and  sufficient  to  do  the  job?  Lastly,  what  level  of 
learner  control  is  advisable?  Users  choose  where  to  go,  but  should  they  have  the  freedom  to  go  anywhere? 

The  purpose  of  the  questions  raised  is  to  attempt  to  systematize  some  of  the  choices  developers  must  make 
when  they  design  hypermedia  databases.  A second  consideration  is  whether  the  hypermedia  applications  created  ten 
years  ago  (when  hypermedia  environments  were  first  became  practical  realities)  can  be  significantly  improved  by 
introducing  those  features  that  arc  currently  being  advocated  by  hypermedia  designers. 

How  Can  Users  Avoid  Becoming  Lost  in  Kyperspace? 

Many  authors  have  agreed  that  a major  problem,  if  not  the  major  problem,  confronting  users  of  hypermedia 
systems  is  becoming  lost  in  hyperspace  (Bernstein,  1991;  Conklin,  1987;  Jonasscn  & Grabinger,  1990;  Local  is. 
Leiourncau,  & Banvard,  1989;  Marehionini.  1988;  Parunak,  1989;  Shneiderman  & Kcarslcy,  1989;  Stanton,  Taylor, 
& Twccdic,  1992).  Stanton  ct  al.  (1992)  pointed  out  there  are  inherent  problems  in  navigating  through  a computer 
world  that  appears  to  be  two-dimensional,  although  the  underlying  structure  or  metaphor  may  be  three-dimensional. 
Picher,  Berk,  Devlin,  and  Pugh  (1991)  argued  that  getting  lost  in  hype xmedia  applications  is  more  likely  than  getting 
lost  in  hyper  text  documents  because  the  rules  for  navigating  through  nodes  containing  video,  pictures,  and  sound  are 
necessarily  more  complex  than  navigating  text  alone.  In  fact,  Bernstein  (1991)  and  Jonasscn  (1989)  contended  that 
users  who  fear  they  arc  overlooking  crucial  information  because  of  navigation  problems  may  abandon  hypermedia 
in  favor  of  regular  materials. 
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Shncidcrman  and  Kcarslcy  (1989)  posited  that  (here  arc  two  aspects  to  being  lost  in  hyperspace:  not  finding 
desired  information  and  gening  disoriented.  They  referred  to  this  problem  as  the  “dark  side”  (p.  10)  of  having  the 
freedom  to  explore  easily  in  hypertext.  Edwards  and  Hardman  (1989)  listed  three  conditions  wherein  users  are 
“lost”:  (a)  not  knowing  where  to  go  next;  (b)  knowing  where  to  go  next,  but  not  how  to  get  there;  and  (c)  not 
knowing  where  they  are  in  the  overall  structure  of  the  document.  Conklin  (1987)  also  cited  the  last  two  conditions 
and  called  them  a “disorientation  problem”  (p.  38).  Bernstein  (1991)  pointed  out  yet  another  faeet  of  disorientation. 
He  maintained  that  an  unexpected  recursion  (return  to  a previously-visited  node)  is  intrinsically  disorienting, 
especially  when  such  recursions  are  rare.  Users  may  perceive  recursion  as  a signal  that  the  database  has  been 
exhausted  or  that  there  has  been  a system  error.  He  explained  that  some  authors. of  hypermedia  novels  such  as 
mysteries  may  want  to  intentionally  disorient  the  reader  by  using  recursion,  hut  that  in  general  unexpected  recursion 
is  to  he  avoided. 

McAleese  (1989)  noted  attempts  for  combating  the  lost-itvhypcrspacc  syndrome  in  two  well-known 
applications.  In  NoteCards,  a Graphical  History  Tree  records  and  displays  the  “neighborhood”  of  i des  that  have 
been  visited  by  the  user.  The  nodes  arc  marked  so  that  users  can  see  which  links  they  have  used.  While  a plain 
History  Tree  displays  all  accessed  nodes  in  an  hierarchical  trail,  a Summary  Tree  is  a special  ease  of  a History  Tree 
where  annotations  made  by  the  users  are  also  displayed.  McAleese  pointed  out  that  HyperCard  has  a Summary 
Window,  that  users  can  activate  by  using  the  Recent  command,  which  displays  as  many  as  the  last  42  distinct 
cards/scrccns  visited,  but  not  the  links  used,  and  allows  users  to  jump  to  any  one  of  those  eards/sereens  by  clicking 
on  its  image.  Fujiliara,  Snell,  and  Boyle  (1992)  applauded  systems  such  as  the  Perseus  Project  (on  ancient  Greek 
culture  at  Harvard  University)  that  have  paths  that  can  be  saved  for  later  use.  Shncidcrman  and  Kcarslcy  (1989) 
contended  that  reversibility  (being  able  to  retrace  steps  and  return  to  previous  screens)  is  important  to  reduce 
disorientation. 

Horton  (1994)  claimed  that  carefully  chosen  icons  accompanied  by  short  word  labels  outperform  controls 
marked  only  by  an  icon  or  a word  label,  not  both.  Tolhurst  (1993)  recommended  the  use  of  n friendly  guide  that 
could  be  activated  by  clicking  on  an  omnipresent  icon  for  the  guide.  The  guide  could  then  make  navigation 
suggestions  to  the  user.  Oren.  Salomon,  Kreihnan,  and  Don  (1990)  showed  a preference  for  what  they  called  the 
“guides  metaphor”  over  the  “navigation  metaphor”  for  helping  users  make  navigational  choices,  even  though  they 
found  that  users  sometimes  confused  the  navigation  advice  given  by  guides  with  the  content  stories  provided  by 
other  entities  that  also  looked  like  navigational  guides.  Kappe,  Maurer,  and  Sherbakov  (1993)  staled  that  since 
typical  solutions  that  work  well  on  small  systems  fail  completely  when  applied  to  large  systems,  navigation  must  he 
extended  by  adding  hierarchical  organization  of  related  material,  guided  tours  through  the  information  prepared  by 
content  experts,  and  advanced  database  query  capabilities. 

Should  Hypermaps  lie  Provided? 

Edwards  and  Hardman  (1989)  identified  two  principal  advantages  to  having  a survey-type  cognitive  map  of 
any  environment,  be  it  a city  ora  database.  First,  there  is  the  opportunity  to  visualize  and  then  to  utilize  short  cuts  to 
reach  desired  locations.  Second,  for  users  who  become  distracted  or  lost  cn  route  to  a specific  goal,  there  is  a far 
greater  chance  that  they  can  regain  their  hearings  and  reach  their  intended  destination  if  they  have  a spatial  cognitive 
map  of  the  environment.  Kelly  (1993)  cautioned  that  navigational  aids  such  as  maps  based  on  spatial  analogies  or 
metaphors  may  he  appropriate  for  spatial  navigation,  but  not  for  conceptual  navigation. 

Scllcn  and  Nicol  ( 1990)  gave  two  other  rationales  for  maps.  They  claimed  that  making  mental  navigational 
models  explicit  within  hypermedia  helps  reduce  the  cognitive  load  for  users,  who  would  no  longer  have  to  rely  on 
their  memories  to  tell  them  where  they  arc.  In  addition,  a map  can  form  the  basis  for  a mental  model  of  how  to  travel 
from  one  point  to  another  in  the  database  without  having  to  keep  in  mind  a set  of  procedures  for  navigation.  Scllcn 
and  Nicol  said  that  the  type  of  map  depends  on  the  nature  and  structure  of  the  application.  Ir,  a file  management 
database,  for  example,  the  map  might  display  the  hierarchical  location  of  a file  within  a folder  structure.  However, 
maps  could  also  take  the  appearance  of  you-arc-iicrc  maps  found  in  public  buildings. 

Tolhurst  (1992)  suggested  that  graphic  environmental  maps  based  on  some  concept  (metaphor  or  mimesis) 
familiar  to  the  user  would  he  a useful  navigational  aid.  For  example,  a learner  studying  human  digestion  could  be 
presented  with  a diagram  of  a body  with  the  various  parts  of  the  digestive  system  in  view.  By  clicking  with  a mouse, 
users  could  choose  body  parts  and  reveal  linked  information  about  that  part  of  the  digestive  system.  At  any  point  in 
the  investigation  of  digestion,  users  could  return  to  the  body  diagram  (map)  to  reorient  themselves  if  necessary. 

Shncidcrman  and  Kcarslcy  (1989)  pointed  out  that  whereas  the  table  ol'  contents  reveals  a book’s  structure 
and  content,  it  is  not  automatically  apparent  in  hypermedia  applications  what  information  is  available,  how  it  is 
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structured,  and  where  the  user  is  now.  For  hypermedia  applications,  they  advised  thai  a hierarchical  view  of 
structure  and  content  he.  displayed  by  means  of  a dedicated  area  of  the  screen  (which  of  course  is  only  possible  in  a 
database  with  hierarchical  structure).  Loeatis  ct  al.  ( 1989)  as  well  as  Shncidcmian  and  Kearslcy  advocated  having  a 
“fish  eye”  view  capability  which  allows  users  to  see  “nearby”  nodes.  Rada  (1995)  claimed  that  "hierarchical  or 
fisheye  views  help  users  get  a sense  of  the  overall  information  landscape"  (p.26). 

A number  of  authors  have  recommended  some  type  of  overall  structure  maps  for  navigational  purposes 
(Conklin,  1987;  Gay  & Mazur,  1991;  Gygi,  1990;  Litchfield,  1993;  Loeatis  el  al.,  1989;  Parunak,  1989;  TolliurM. 
1992).  Parunak  (1989)  claimed  that  useful  maps  could  he  defined  even  in  large  hypermedia  systems  as  long  as  what 
he  called  the  “topology,"  the  system  of  links,  was  appropriate. 

However,  Landow  (1990)  declared  that  global  maps  do  not  work  for  any  but  the  smallest  of  hypermedia 
databases.  Landow  drew  this  conclusion  from  his  work  with  Intermedia  (a  hypermedia  application  generator 
originated  at  Brown  University  where  Landow  worked).  Landow  held  that  navigation  is  not  a major  problem  in 
Intermedia  since  system  features  allow  users  to  locate  and  travel  to  any  information  in  the  database  by  full  text 
searches,  folders,  links,  web  views,  and  menus  of  link  choices.  He  stated  that  in  Intermedia  users  always  know  what 
documents  “surround”  the  one  being  read  through  the  use  of  a local  tracking  map  (the  web  view)  and  that  users  can 
always  travel  to  an  overview  document  that  helps  reorient  them. 

Stanton  cl  al.  (1992)  conducted  a study  with  one  hypothesis:  ‘The  presence  of  a spatial  map  will  improve 
navigation  and  support  cognitive  map  formation”  (p.  433).  The  study  was  conducted  using  a relatively  small 
hypertext  environment  with  42  screens.  The  study  used  local  (fish  eye)  maps.  Contrary  to  other  studies.  Stanton  et 
al.  found  that  the  presence  of  maps  resulted  in  poorer  performance.  In  addition,  participants  using  maps  reported  a 
perception  that  they  had  significantly  less  control  over  the  information  in  the  database  than  that  reported  by  non-map 
participants.  The  non-map  participants  also  performed  significantly  better  than  the  map  participants  on  their  ability 
to  classify  the  screens  on  a cognitive  map  task.  Stanton  ct  al.  concluded  that  map  aids  may  reduce  the  need  for  or 
interfere  with  learners’  actively  constructing  their  own  cognitive  maps  — that  it  is  wrong  to  assume  that  a map  will 
always  aid  performance. 

What  Is  the  Proper  Balance  between  Links  and  Nodes? 

Glushko  (1989)  claimed  that  excessive  linking  causes  serious  problems  of  disorientation  and  cognitive 
overload  for  the  user  because  it  destroys  most  of  the  structural  and  contextual  cues  that  users  rely  on  for  navigation. 
Although  he  stated  that  several  researchers  are  working  on  creating  new  hypertext  linking  structures,  he  staled  the 
opinion  that  limiting  the  links  in  the  first  place  was  a more  practical  solution.  Langston  and  Grosser  (1993) 
described  one  such  system  with  a Point  and  Query  interface  where  navigation  is  accomplished  by  combining  a list  of 
subjects  with  a list  of  questions.  They  claimed  that  users  benefit  from  the  “internodal  coherence”  (p.  357)  that  results 
from  following  such  question  trails. 

McAleese  (1989)  described  the  “net”  metaphor  for  hypertext  linkages.  He  explained  that  in  a “true  network 
system"  (p.  14)  any  given  node  is  at  most  one  link  away  from  any  other  node,  and  that  no  node  is  more  important 
than  any  other  node.  He  termed  network  systems  the  ideal  solution  for  understanding  the  linkages  in  hypertext. 
However,  he  stated  that  structuring,  constraining  the  user's  access  to  other  nodes,  was  important  if  the  system 
browser  was  to  provide  the  user  with  scaffolding  for  purposeful  navigation.  Conklin  (1987)  envisioned  users 
navigating  databases  wa  three  modes:  by  using  links,  by  searching  lor  keywords  (objects  with  certain  attributes  such 
as  strings  of  characters  or  images),  and  by  using  a browser  that  displays  the  network  graphically.  Hence,  Conklin's 
description  of  hypermedia  supported  both  the  network  system  approach  and  the  structured  or  constrained  approach 
to  navigation. 

What  Level  of  Learner  Control  Is  Advisable? 

Litchfield  (1993)  defined  learner  control  to  be  the  amount  of  latitude  a learner  has  over  the  direction  and 
depth  of  investigation.  She  claimed  that  for  some  users  too  much  learner  control  is  counterproductive,  and  that  many 
users  choose  to  return  to  programs  where  the  content  is  program  controlled.  On  the  other  hand,  Shuman  (1998) 
asserted  that  hypermedia  empowers  users  by  giving  them  control  — control  that  allows  users  to  individualize  their 
own  instruction  to  address  different  learning  styles  and  needs.  Bernstein  (1991)  saw  both  sides  of  the  issue.  On  the 
one  hand,  he  said  hypermedia  authors  could  acknowledge  learners'  active  participation  by  giving  them  a richer  web 
of  links  to  traverse.  However,  he  said  an  author  could  reduce  the  apparent  complexity  of  the  content  for  learners  by 
imposing  a clear  and  limited  organizational  scheme  on  an  otherwise  complex  system. 
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Shneiderman  and  Kearslcy  (1989)  reported  that  some  interface  researchers  hold  the  opinion  that  building 
sophisticated  “intelligent  agents”  undermines  the  users’  sense  of  control  and  their  feelings  of  accomplishment. 
Instead  of  building  intelligence  into  the  machine,  the  designers  seek  to  enhance  learner  control  by  providing 
navigation  tools  for  browsing,  searching,  and  employing  filters.  Laurel  (1990)  countered  this  argument  by  saying 
that  a good  agent  "will  do  what  1 want,  tel!  me  all  1 want  to  know  about  what  it’s  doing,  and  give  me  back  the  reins 
when  1 desire”  (p.  357).  She  proposed  that  only  users  who  wanted  to  use  agents  should  have  them,  and  those  who  do 
not  should  have  other  choices. 

Jonasscn  and  Grabingcr  (1990)  maintained  that  learner  control  permits  the  user  to  direct  the  sequence  of 
instruction  in  such  a way  that  the  user  obtains  the  appropriate  type  and  amount  of  support.  Instead  of  the  instruction 
directing  learners,  learners  should  he  empowered  to  make  judicious  choices  to  adapt  the  instruction  to  their  own 
needs  and  interests.  For  Jonasscn  and  Grabingcr,  this  notion  is  supported  by  the  theory  that  learners  know  what  is 
best  for  them  and  that  learners  in  control  of  instruction  will  invest  more  mental  effort  in  their  learning.  Nevertheless, 
Jonassen  and  Grabingcr  admitted  that  research  does  not  support  this  sentiment,  especially  for  average  and  hclow- 
average  learners  — that  learners  do  not  make  the  best  decisions  when  given  unrestricted  control. 

Learning  Theories  Implemented  in  Hypermedia  Applications 

Sonic  authors  have  associated  the  type  of  navigation  in  various  systems  with  particular  learning  theories. 
Horncy  (1993)  said  that  the  network  system  approach,  where  each  node  is  linked  to  every  other  node,  supports 
constructivist  learning.  By  contrast,  hypermedia  applications  that  exhibit  linear  or  severely  constrained  link 
structures  could  be  construed  to  be  bchaviorist  in  philosophy.  Traditional  cognitive  theories  would  then  he 
associated  with  applications  situated  between  the  two  extremes.  Under  this  scheme,  bchaviorisls  would  favor  linear 
navigation  with  limited  learner  control  and  no  maps.  Constructivists  would  favor  unrestricted  navigation  with  high 
learner  control  and  hypermaps  that  would  aliow  users  to  jump  easily  from  place  to  place  in  the  database.  Convinced 
that  individuals  arc  more  than  flesh-and-blood,  stimulus-response  machines  and  armed  with  the  results  of  studies 
that  show  unrestricted  travel  between  nodes  to  be  disorienting  for  all  but  the  most  gifted  of  learners,  traditional 
cognitivists  would  chart  a central  course  with  many  opportunities  for  non-linear  exploration.  Cognitivists  would 
choose  to  have  learner  control  high  but  not  so  high  that  it  creates  disorientation  and  distraction.  Cognitivists  would 
also  take  advantage  of  cognitivist  devices  like  maps  and  metaphors  to  supply  navigation  clues  for  users.  Most 
hypermedia  applications  fall  under  the  cognitivist  rubric.  Readers  may  find  that  the  approaches  used  and  conclusions 
drawn  about  hypermedia  environments  arc  best  understood  by  considering  the  associated  learning  theories  that  the 
authors  advocate. 

Research  Questions 

The  issues  discussed  above  suggest  the  following  research  questions: 

1 . What  is  the  appropriate  screen  layout  for  navigation  in  a hypermedia  database? 

a.  Should  navigation  controls  reside  in  dedicated  areas  or  he  available  only  when  requested  by  users? 

b.  What  arc  the  most  effective  representations  for  navigation  controls?  For  example,  should 
navigation  controls  be  marked  by  icons,  by  names,  or  both? 

c.  Should  navigation  controls  overlap  other  controls  on  the  screen? 

2.  What  is  the  appropriate  use  of  color  in  a hypermedia  database? 

a.  Should  special  colors  be  used  to  allow  users  to  differentiate  between  certain  types  of  navigation 
controls  ? 

b.  Should  special  colors  be  used  to  allow  users  to  differentiate  between  various  locations  in  the 
database? 

3.  What  is  the  appropriate  use  of  hypermaps  or  other  locational  devices? 

a.  Should  users  be  provided  with  hypcrniaps  or  other  devices  so  that  they  have  a sense  of  where  they 
arc  or  have  been? 

b.  Should  hypermaps  or  other  location-indicating  devices  also  allow  users  to  jump  from  place  to 
place  in  the  hypermedia  database? 

c.  Should  some  device  be  provided  to  tell  users  how  "deep”  they  are  in  the  database? 

4.  What  is  the  appropriate  degree  of  consistency  to  use  in  the  design  of  navigation? 

a.  Should  the  functions  of  icons  be  consistent  throughout  the  database? 

b.  Should  controls  be  consistent  as  to  location  on  the  screen,  what  they  look  like,  and  what  happens 
when  clicked? 
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5.  What  arc  the  appropriate  types  of  pathways  for  transit  through  a hypermedia  database? 

a.  To  what  extent  should  users  be  able  to  retrace  their  steps  or  save  paths  made  through  the  database? 

b.  How  easy  should  it  be  for  users  to  access,  alter,  or  digress  from  previously  stored  paths  through 
the  database? 

6.  What  is  the  appropriate  level  of  user  control  in  a hypermedia  database? 

a.  To  what  extent  should  expert  users  be  able  to  move  through  the  database  quickly  with  fewer  steps 
or  key  strokes  than  would  be  prudent  for  novice  users? 

b.  Is  it  better  to  have  a few  long  pull-down  menus  or  several  shorter  ones? 

c.  Should  designers  eliminate  controls  that  most  users  arc  not  expected  to  use  (that  may  be  accessed 
in  some  alternate  manner  by  those  who  need  them)? 

d.  To  what  extent  should  users  be  able  to  wander  through  the  database  to  see  what  is  available  (the 
way  that  one  might  browse  through  a set  of  encyclopedias)? 

c.  What  mechanisms  should  designers  provide  so  that  users  can  determine  with  some  certaimy 
whether  or  not  a particular  piece  of  information  is  present? 

7.  What  is  the  appropriate  use  of  a help  function  in  a hypermedia  database? 

a.  To  what  extent  should  the  help  function  be  sensitive  to  the  context  when  help  is  sought  by  users? 

b.  To  what  extent  should  the  help  function  anticipate  problems  that  users  may  have  in  using  the 
database  and  suggest  solutions? 

c.  To  what  extent  should  the  help  function  suggest  strategics  for  using  the  database? 

8.  What  is  the  appropriate  relationship  between  the  learning  styles  of  users  and  the  navigational  controls 

available  in  a hypermedia  database? 

Methodology  for  Proposed  Research  on  Hypermedia  Navigation 

As  a strategy  for  resolving  the  research  questions  listed  above,  one  docs  not  necessarily  need  to  create  a 
brand  new  hypermedia  database.  One  can  examine  whether  the  hypermedia  databases  created  in  the  past,  often  a* 
great  expense,  can  be  significantly  improved  by  introducing  those  navigational  features  that  arc  currently  being 
advocated  by  hypermedia  designers.  One  such  database  is  the  Perseus  Project.  Hypermedia  applications  were  first 
developed  for  personal  computers  some  ten  years  ago  (Hall,  Davis,  & Hutchings,  1996).  The  first  version  of  the 
Perseus  hypermedia  database  embodied  the  design  principles  employed  at  that  time.  The  second  and  most  recent 
version  of  Perseus  released  in  1996  still  retains  the  original  navigational  design  structure  proposed  in  1988.  Since 
Perseus  is  a hierarchical  database,  it  is  possible  to  alter  the  overall  navigational  superstructure  in  order  to  test  various 
hypotheses  about  navigational  design  without  changing  the  vast  majority  of  the  information  stored  in  the  database. 

The  Perseus  Project 

The  Perseus  Project  is  an  ambitious  undertaking  begun  in  the  Classics  Department  of  Harvard  University.  It 
is  devoted  to  the  creation  of  a massive  interactive  hypermedia  application,  called  “Perseus,”  on  ancient  Greek 
culture.  According  to  Crane  and  Mylonas  in  their  1988  article:  “The  Perseus  Project  is  developing  interactive, 
computer-based  materials  on  Greek  Civilization.  These  arc  designed  to  support  learners,  instructors,  and  researchers 
as  they  explore  this  complex  subject'1  (p.  25).  The  initial  four-year  funding  proposal  for  Perseus  to  the 
Anncnbcrg/CPB  Project,  Apple  Computer,  and  Harvard  University  dated  May  24,  1988,  was  for  $3,390,541,  with 
the  final  project  cost  between  $4  and  $5  million  (Hughes,  1988).  The  literature  about  hypermedia  applications 
includes  frequent  favorable  references  to  Perseus.  It  has  been  praised  in  journal  articles  as  a model  for  hypermedia 
development.  Hughes  called  the  Perseus  Project  “today’s  most  ambitious,  intriguing,  and  promising  application  of 
computer  technology  to  academic  insh  ction  and  research  in  the  liberal  arts”  (1988,  p.  I).  In  her  article  about 
evaluating  Perseus,  Neuman  asserted:  “The  project  is  innovative  in  its  development  process,  its  technological 
sophistication,  its  range  of  potential  applications,  and  its  intended  outcomes”  (1991,  p.  239).  Since  its  carefully 
developed  design  epitomizes  the  navigational  controls  typical  of  hypermedia  databases  created  over  ten  years  ago 
(See  Figures  1 and  2 for  examples),  it  is  a fit  subject  for  the  intended  research. 
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Figure  /.  The  Floating  Perseus  Navigator  in  Perseus  2.0 


Figure  2.  The  Perseus  Gateway 


Previous  Evaluation  Results 

Marchionini  and  Crane  ( 1 994)  reported  the  results  of  a three-year  evaluation  of  Perseus  in  a lengthy  article 
in  the  ACM  Transactions  on  Information  Systems.  The  evaluation  examined  a number  of  instructional  design  issues. 
However,  the  evaluation  was  summativc,  not  formative,  in  nature.  Also,  the  evaluation  was  qualitative  (naturalistic 
inquiry),  not  quantitative.  Rather  than  suggest  changes  to  the  Perseus  application,  Marchionini  and  Crane  suggested 
how  users  could  be  better  prepared  or  trained  to  use  Perseus  more  effectively. 
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The  evaluation  effort  described  here  involves  a complex  system  (Perseus)  based  upon  a new  technology 
(hypermedia)  applied  to  abstract  goals  (finding  relevant  information,  learning  how  to  think  critically).  The 
overall  evaluation  effort  aims  to  inform  general  hypermedia  application  design,  develop  human-computer 
interaction  and  information-seeking  behavior  theories,  and  add  to  our  understanding  of  learning  and 
teaching.  (Marchionini  and  Crane,  1994,  p.  6) 

The  Proposed  Research  Study 

Testing  the  long  list  of  research  questions  given  above  is  a daunting  task  by  anyone's  standards.  Even  if 
only  two  alternative  hypotheses  arc  consider.  J for  each  of  the  eight  categories  of  research  questions  above,  sixty- 
four  test  groups  would  be  required  to  consider  all  possible  interactions  between  the  various  treatments.  Hence,  the 
proposed  study  will  concentrate  on  two  treatments:  decentralized  dedicated  areas  for  navigational  controls  vs.  the 
centralized  menu  driven  design  of  the  original  Perseus;  and  unique,  named  iconic  representations  of  navigational 
controls  vs.  the  representations  in  the  original  Perseus. 

Hence,  there  will  be  four  test  groups  in  this  two  by  two  design:  (a)  the  control  group,  which  will  use 
Perseus  in  its  original  form,  (b)  a group  tested  with  Perseus  modified  to  have  dedicated  areas  for  navigational 
controls,  (c)  a group  using  Perseus  modified  to  have  unique  named  iconic  representations  for  navigational  control, 
and  (d)  a group  using  Perseus  modified  to  have  both  dedicated  areas  and  unique  named  icons  (See  Figure  3). 
Participants  for  each  group  will  be  randomly  selected  from  the  target  population,  college-level  students  with  some 
hypermedia  experience. 

Participants  will  be  given  a pretest  and  posttest  to  measure  comprehension.  They  will  be  tested  individually 
using  the  speak-aloud  protocol  while  they  are  using  the  Perseus  hypermedia  database.  Each  session  will  be 
videotaped  to  preserve  a record  of  each  participant's  performance  (verbal  and  physical)  using  the  version  of  Perseus 
appropriate  to  the  designated  group  for  that  individual.  The  testing  will  produce  two  types  of  data:  (!)  measurements 
for  comprehension  by  use  of  pencil  and  paper  testing  before  and  after  each  session,  and  (2)  measurements  of  how 
long  it  takes  to  accomplish  specified  navigational  tasks.  The  sessions  themselves  will  uc  "scavenger  hunts"  through 
the  Perseus  database  for  the  answers  to  a set  of  approximately  ten  questions.  The  sessions  are  called  scavenger  hunts 
because  the  questions  may  be  answered  in  any  order,  thus,  simulating  a hypermedia  learning  environment  where  one 
might  screndipitously  stumble  upon  one  answer  while  actually  looking  for  another. 

Future  Directions 

Assuming  that  the  research  study  reveals  a significant  difference  between  the  treatment  groups  and  the 
control  group  in  the  proposed  study,  the  authors  intend  to  continue  testing  the  various  navigational  research 
questions  given  above.  The  ultimate  goal  would  be  to  develop  a navigational  system  for  Perseus  that  incorporates  ail 
the  navigational  features  that  produced  significant  improvement  in  comprehension  and  efficient  usage  (speed). 
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Figure  3.  Suggested  Decentralized  Navigational  Design  to  Replace  the  Perseus  Gateway 
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Abstract 

Building  a community  of  thinkers  is  a goal  for  education.  In  a world  in  which  technology  provides  a means 
to  communicate  quickly  and  efficiently,  educators  need  to  develop  models  for  the  use  of  technology  as  an  integral 
part  of  building  an  environment  for  teachers  to  thinking  critically  and  creatively. 

Technology  was  used  to  help  create  a community  of  thinkers  in  the  Social  Studies  in  the  Elementary  School 
course  at  the  University  of  Missouri-Columbia.  During  the  preservice  course,  students  used  technology  to  write  and 
think  critically  about  social  studies  education.  Through  on-line  reflective  journals,  electronic  mail,  a class  listerv,  a 
newsgroup , and  the  Internet,  students  incorporated  technology  into  their  research,  writing,  reflecting  and  reacting 
to  social  studies  in  the  elementary  school.  Technology  supported  a forum  for  open  communication  between  students 
and  instructors  and  enriched  the  learning  environment.  By  building  technology  into  the  pedagogy  of  the  course , it 
provided  a means  for  building  a community  of  thinkers. 

To  create  a community  of  thinkers  in  the  classroom  everyone  should  be  responsible  of  creating  ideas  and 
sharing  knowledge.  Students  were  the  best  advocated  for  promoting  the  integration  of  technology  into  learning 
process  and  they  can  be  held  responsible  to  the  group  for  participating  in  the  thinking,  writing  and  technology.  It 
was  the  instructor's  responsibility  to  determine  when  the  quality  of  the  task  would  be  enhanced  with  the  use  of 
technology.  The  integration  of  technology  into  the  classroom  provided  the  students  with  more  time  for  quality 
reflection  and  made  better  use  of  classtime. 

Technology:  Creating  a Community  of  Thinkers 

A challenge  of  colleges  and  universities  for  the  21st  century  is  Ihe  integration  of  technology  into  the 
learning  environment  (Barksdale  1996,  Jonassen  1995,  Snadcll,  Stewart  & Stewart  1996).  By  incorporating 
technology  into  the  college  classroom  the  students  expand  their  technological  literacy  and  expand  the  learning 
environment  beyond  the  classroom  walls.  Technology  is  not  only  a tool  for  receiving  and  disseminaing  ideas, 
technology  can  be  a means  for  articulating  ideas,  support  for  differing  viewpoints,  and  responding  in  thoughtful  and 
creative  ways.  The  use  of  telecommunication  bonds  ihc  participants  in  interdependence,  congruence  of  the  task  and 
provides  a forum  for  reflection  (Thomas,  Clift,  & Sugimolo  1996) 

Technology  must  he  a integral  part  of  the  pedagogy  of  leaching  and  learning  environment.  A meaningful 
learning  environment  includes  the  following  seven  qualities  active,  constructive,  collaborative,  conversational, 
intentional,  contcxtualiz.cd,  and  reflective  (Jonassen  1995)  and  technology  can  be  a partner  in  meaningful  learning. 
Educators  need  to  find  ways  to  promote  a constructivist  learning  environment  and  a community  of  learners,  in  which 
students  are  active  in  selecting  what  to  learn,  processing  new  knowledge,  and  assessing  his/her  progress  and  using 
technology  as  a means  to  support  the  environment 

The  paper  will  include  how  the  social  studies  in  the  elementary  school  course  developed  a community  of 
learners  and  incorporated  technology.  Several  methods  for  using  technology  to  facilitate  thinking  in  the  social 
studies  education  course  were  (1)  1 istscrv,  (2)  electronic  mail,  (3)  a newsgroup,  (4)  on-line  reflective  journals,  (5)  in 
class  on-line  assignments,  and  (6)  implementation  of  technology  into  elementary  schools  lessons. 

Setting 

Since  the  1996  fall  semester,  there  have  been  over  180  students  in  the  Social  Studies  in  the  Elementary 
School  course.  For  course  assignments,  most  of  the  students  u^ed  the  Macintosh  or  PC  computers  in  the  Reflector, 
the  College  of  Education  computer  laboratory.  As  students  of  (he  university,  they  have  free  access  to  electronic  mail, 
the  Internet,  course  newsgroup  and  a class  lislscrv.  The  instructors  provided  an  introduction  to  the  use  of  each  of  the 
forms  of  communication  that  is  used  in  the  course.  Wc  provide  the  e-mail  address  of  each  individual  in  the  course, 
the  class  newsgroup  and  lislscrv  addresses,  and  the  home  page  for  course. 


Social  Studies  in  the  Elementary  School  is  a required  three  semester  hour  course  for  the  undergraduate 
elementary  teacher  education  students  at  the  University  of  Missouri-Columbia.  All  course  information  is  posted  on 
the  course  home  page:  http://www.coejnissouri.edu/-cssc.  The  course  is  technology  based  and  meets  the 

university’s  writing  intensive  requirements.  Within  the  writing  assignments  for  the  course,  students  are  required  to 
write  drafts  of  their  work,  revise  and  resubmit  work,  incorporate  critical  thinking  into  writing  and  write  more  than 
20  pages  during  the  term.  The  course  is  scheduled  for  50  minutes  on  Monday,  Wednesday  and  Friday  and  the  Friday 
sessions  arc  conducted  in  a computer  laboratory. 

Technology  Related  Assignments 


Assignments 

Technology 

In-class  Group  Work 

Li  st  sc  rv 

Internet  Exploration 

Internet 

Software  Review 

CD  Roms 

Discussion  Groups  on  Issues 

Listserv 

Reflective  Journals 

On-line  form 

Field  Experiences 

Varied 

Article  Review 

Newsgroup 

Concept  Maps 

PIVIT 

Graphic  Design 

Photo  Shop 

^Excluding  word  processing  for  writing. 

The  environment  of  the  course  is  established  with  the  focus  on  developing  a community  of  thinkers.  An 
environment  that  incorporates  computers  needs  to  have  a pedagogical  style  which  utilities  the  instructor  as  a 
facilitator  and  a collaborative  style  of  instruction  (Goodson  & Mangan  1991).  The  instructor  is  a facilitator  for 
learning  and  the  students  work  as  a community  of  learners  by  completing  the  assignments  as  a team  of  preserviee 
teachers.  The  class  meets  in  a traditional  classroom,  a computer  laboratory  and  in  student  selected  locations. 

In  on-line  reflective  journals  the  students  have  made  comments  and  reactions  to  the  course  design  and  the 
role  of  technology  in  the  development  of  preservice  teachers. 

I keep  plugging  away.  At  least  now  I don’t  find  it  scary,  I find  it  challenging. 

I'm  glad  we  had  a chance  to  do  this — it  was  learning  through  discovery! 

Electronic  Mail 

Electronic  mail  (e-mail)  is  defined  as  an  electronic  communication  system  that  enables  users  to  share 
information  about  topics  of  common  interest  (Voekcll  and  Brown  1999).  A subject  of  joint  interest  to  ail  students  are 
class  assignments.  Using  e-mail,  the  instructor  possesses  the  ability  to  inform  all  students  of  upcoming  class  events. 

A typical  collegiate  class  possesses  many  academic  deadlines.  Assignment  are  due  at  a certain  time  and 
student  stress  rates  elevate  as  completion  dates  draw  near.  When  the  students  are  feeling  anxious  about  their  work 
the  instructor  must  assume  the  role  of  a motivator.  Students  received  these  messages: 

You  should  be  very  proud  of  your  work  on  the  clusters  this  term.  I hope  you  can  take  advantage  of  your 

work  when  you  student  teach. 

E-mail  is  a form  of  communication  in  which  we  can  learn  from  each  other. 

Electronic  mail  allows  students  “to  send  messages  to  individual  colleagues,  groups”  (Buckley  1995)  or  the 

entire  class  at  one  time.  One  sludcni  wrote  to  the  collective  group  of  elementary  social  studies  students: 
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My  group’s  cluster  project  is  going  a!!  right.  We  arc  doing  endangered  species,  and  it  is  proving  an  easy 
topic  to  l'ind  information  about.  Good  luck  to  anyone  that  is  presenting  their  project  in  the  next  week. 

The  implementing  of  technology,  in  a class  curriculum,  can  cause  fear  for  many  students.  E-mail  can  he 
used  to  case  these  feelings  of  inadequacy.  By  employing  e-mail  students  “vented”  their  frustrations  concerning 
technology  usage. 

I didn’t  know  how  to  use  the  Internet  before  this  class.  1 was  almost  terrified  when  it  came  to  using 
computers. 

The  one  thing  l would  like  to  mention  is  that  I’m  not  afraid  of  technology  and  I am  confident  that  1 can 
learn. 

As  the  students  gained  confidence  in  their  ability  to  use  technology  as  a communication  tool,  e-mail 
responses  reflected  the  students  new  found  confidence: 

As  an  elementary  social  studies  teacher  I will  bring  technology  into  my  classroom  by  making  it  available  to 
students  and  using  it  in  planning  and  carrying  out  lessons. 

I use  e-mail  a lot  more  often  than  I used  to.  I also  gained  great  ideas  from  classmates  because  of  the 
listserv. 

It  was  very  beneficial  to  use  technology  in  order  to  get  used  to  it  before  stepping  into  the  classroom.  It  has 
prepared  me  to  use  technology  in  the  future  and  I will  be  more  able  to  apply  my  skills. 

Electronic  mail  enhances  the  instructor’s  ability  to  form  a community  of  learners.  Electronic  mail 
encourages  opportunities  for  meaningful  reflection  and  allows  the  student  and  instructor  to  observe  the  thinking  and 
construction  of  the  meaning  (Mclntryc  and  Tiusty  1995).  When  students  have  the  motivation  to  use  and  the 
knowledge  to  operate  e-mail,  a classroom  of  strangers  becomes  a grouping  colleagues.  In  this  scenario,  SHARED 
information  is  the  power  that  builds  professional  unity 

Discussion  Lists 

Establishing  a community  of  learners  was  accomplished  in  the  implementing  of  a class  listserv.  The 
listserv,  designed  as  a virtual  forum  (Wild  1996)  for  class  participation,  facilitated  the  transfer  of  ideas  between 
students  and  instructors.  Involvement  in  listserv  activities  was  established  by  the  organizational  framework  of  the 
listserv. 

The  class  subscribed  to  the  listserv  following  a classroom  discussion  concerning  the  educational 
possibilities  of  listserv  usage  in  elementary  school  social  studies  classes.  Once  the  class  had  joined  the  listserv,  the 
class  was  divided  into  thirds.  Each  student  was  assigned  a number  one.  two  or  three.  Correspondingly,  the  three 
instructors  of  the  class  were  assigned  a number  group  to  cultivate.  On  a rotating  weekly  basis  an  instructor  posed  a 
question  concerned  with  some  aspect  of  elementary  school  social  studies  or  classroom  teaching  situation.  After 
reading  the  listserv  question  the  members  of  the  assigned  group  would  respond  to  the  question.  All  members  of  the 
class  could  respond  to  the  query  of  the  instructor.  Mo  limit  on  response  length  was  determined  by  the  instructors  or 
the  students.  The  students  received  questions  on  Monday  and  were  expected  to  respond  to  the  questions  by  Friday 
of  that  same  week.  A sample  question  the  students  responded  to  dealt  with  student  relations  in  the  classroom. 

You  are  silting  at  your  classroom  desk.  Several  students  are  making  belittling  comments  toward  another 
student.  The  students  feelings  arc  hurt.  The  student  catches  your  eye  and  beseeches  for  help.  What  course 
of  action  do  you  follow  to  change  this  negative  classroom  situation? 

Specific  topics  discussed  on  the  class  listserv  varied.  In  particular,  holidays  were  a popular  discussion 
topic.  The  celebration  of  Halloween,  Christmas  and  Easter  elicited  pointed  responses  among  the  students.  One 
student  wrote  about  observing  Christinas  holiday  in  public  schools: 


The  only  holidays  I remember  learning  ahoul  in  elementary  school  arc  Thanksgiving  and  Christmas.  It 
didn't  bother  me  that  we  were  learning  about  them  because  my  family  celebrates  those  holidays.  Since  there 
arc  so  many  diverse  backgrounds  in  the  classroom,  teaching  holidays  can  he  tricky  and  touchy. 

A student’s  ability  to  express  personal  beliefs  and  thoughts,  with  anonymity,  is  one  of  the  positive  aspects 
of  the  listscrv.  When  asked  to  give  input  upon  “everyday"  classroom  situations,  e-mail  allowed  the  college  students 
to  express  their  feelings  and  supply  feedback  to  the  instructor. 

On  Divorce:  The  topic  of  divorce  is  one  that  is  very  important  to  me,  being  a child  in  a single  parent  home. 

I remember  in  elementary  school  we  had  an  hour  or  two  a month  where  children  of  divorce  could  go  and 
talk  about  their  needs  and  concerns. 

On  Death  in  the  Family:  When  dealing  with  the  death  of  a parent  I would  first  invite  that  student  to  let  me 
ki'Hiw  how  he  or  she  is  feeling.  1 would  share  the  story  of  when  my  mother  died  and  how  1 felt  and  still  do. 

On  Medical  Problems:  As  someone  with  a medical  problem  I can  share  my  experience  on  how  it  was  for 
me.  1 was  diagnosed  with  a heart  condition  at  a very  early  age.  I was  never  able  to  participate  in  activities 
that  my  classmates  did  and  they  never  told  me  why. 

The  disadvantages  of  constructing  a lislserv  for  the  class  were  few.  The  largest  concern  was  the  sheer  bulk 
of  correspondence  (Pearson  1996).  Typically  an  instructor  or  a student  could  respond  to  one  hundred  and  fifty  c- 
mail  messages  during  the  semester.  The  advantages  the  listscrv  presents  for  a class  outweighs  the  concerns  a listscrv 
presents  for  instructors  and  students.  Long  and  complex  conversations  develop  on  listservs,  as  students  explore  their 
developing  understandings  of  both  content  and  pedagogy  (Piburn  & Middleton  1997).  Pearson  (1996)  states 
listservs  establish  the  framework  for  networking  and  scholarly  cooperation,  knowledge  acquisition  a sense  of 
communion  and  an  opportunity  to  keep  pace  with  innovation.  Most  important,  students  find  that  listservs  arc  fun; 
and  they  participate  in  them  with  enthusiasm  (Pearson  1996). 

The  Newsgroup 

Newsgroups  can  be  more  than  a place  to  receive  and  disseminate  information.  Students  can  select  what 
messages  arc  worthy  of  their  response.  The  responses  can  be  a dialogue  of  diverse  points  of  view  and  creative  and 
critical  responses. 

During  the  first  week  of  the  semester,  the  students  subscribe  to  a class  newsgroup  where  messages  arc 
posted.  The  newsgroup  is  attached  to  the  university  electronic  communication  lines  so  a student  can  receive  e-mail 
and  newsgroup  messages  on  the  same  system. 

Each  student  selects  two  exemplary  articles  on  current  issues  in  elementary  school  social  studies  such  as 
religion,  integration,  law  related  education,  holidays,  the  environment,  or  economies.  The  student  should  review' 
journals  such  as  Social  Education.  The  Social  Studies  or  Social  Studies  & the  Young  Learner.  The  article  review 
consist  of  an  APA  reference  for  article,  a paragraph  summary  of  the  article  and  a reaction  to  the  article  with 
discussion  questions  and  the  students  name.  The  students  arc  given  two  weeks  to  submit  each  article  review  on  a 
separate  message  to  the  newsgroup. 

After  everyone  has  submitted  the  articles  reviews  on-line,  each  student  selects  two  article  to  read,  review 
and  respond  to  on-line.  The  response  should  include  the  name  of  the  article,  the  students  name  and  responses  to  the 
questions,  new  ideas,  or  a continuation  of  the  review.  The  students  reply  to  the  message  and  latest  responses  become 
the  last  messages  on  the  newsgroup  list.  The  students  have  two  weeks  to  complete  their  article  responses. 

This  assignment  was  designed  to  incorporate  the  qualities  of  meaningful  learning  (Jonassen  1995)  and 
technology  was  used  to  support  experiences  that  engage  students  in  purposeful  learning.  The  newsgroup  provided  a 
learning  environment  for  students  to  be  actively  engaged,  communicate  and  collaborate  with  peers,  construct 
knowledge  and  reflect  on  ideas  and  issues  in  social  studies  education.  The  assignment  required  the  students  to 
construct  new  knowledge  from  the  information  found  in  professional  social  studies  journals  and  to  actively 
communication  their  ideas  and  reactions  on  the  article  to  their  peers.  The  article  reactions  and  responses  allowed 
students  to  communicate  ideas  to  the  group  and  reflect  on  the  topic.  By  having  the  article  reviews  on  the  newsgroup, 
the  students  read  and  reflected  on  a wide  range  of  current  journal  article  and  topics  that  due  to  time  could  not  be 
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discussed  during  classlimc  or  distributed  in  paper  form.  One  student  said:  “I  really  liked  having  things  posted  in  the 
newsgroup  and  on  the  e-mail.  I eould  check  them  anytime  of  the  day  or  night.1’ 


On-line  Reflective  Journals 

Electronic  journal  writing  models  behavior  that  engages  the  computer  as  a intellectual  partner  in  facilitating 
a specific  task  (Anders  & Brooks  1994).  The  task  can  be  to  write  about  the  topics  for  the  course,  the  assignments, 
issues  in  the  field,  concerns  about  the  course,  rcllcct  on  the  past  and  make  connections  to  the  future.  By  writing  in 
the  journals  the  students  arc  thinking  critically,  reflecting  and  reacting.  The  students  use  the  reflective  journal  to 
actively  construct  new  knowledge. 

The  students  complete  a weekly  on-line  weekly  reflective  journal 
http://www.coe. missouri.cdu/~cssc/reflectivc.hlml.  The  purpose  of  the  journal  is  to  rcllcct  on  an  aspect  of  social 
studies  education  or  the  content  of  the  course  during  the  week.  The  students  select  the  topic  and  focus  of  the 
reflective  journal  entries.  Each  week  the  students  write  a response  to  five  open  ended  statements. 

• What  I learned 

• What  I would  like  to  learn 

• How  will  1 use  what  I have  learned  in  my  future  classroom 

• Goals  for  the  future 

• Common  ts/suggestions 

Upon  completing  the  form,  the  student  submits  his/her  response.  The  entries  for  each  student  arc  filled  in 
an  on-line  data  base  that  contained  a running  log  of  his/her  weekly  responses.  The  journal  entries  can  be  reviewed 
by  the  instructor  or  the  students  to  rcllcct  on  the  course.  The  instructor  read  the  journals  on  a weekly  bases  and 
responded  to  individuals  by  e-mail,  or  to  the  group  as  a listscrv  message  or  as  an  in  class  announcement.  The 
students  can  review  weekly  journals  entries. 


TABLE  1 Sample  Summary  of  Reflective  Journals  for  Fall  1997 


Dale 

Focus 

Summarv/Example 

Sept.  16-20 

Procedural 

Internet 

Oct.  7-11 

Pedagogy 

Field  Experience 

Oct.  21-25 

Pedagogy 

CD  ROMs 

Nov.  4-8 

Diverse 

Unit  Project 

Dee.  2-6 

Pedagogy 

Semester  Closure 

We  found  several  positive  aspects  of  using  an  on-line  reflective  journal  to  support  a meaningful 
environment  and  reflective  learning.  For  reflection,  the  journals  provide  the  students  with  an  opportunity  to 
articulate  what  they  learned  and  rcllcct  on  their  knowledge  and  skills.  Because  the  reflective  journal  was  on-line,  the 
students  select  where  and  when  to  complete  the  journal,  the  students  use  as  much  time  as  needed  to  complete  the 
journal  and  extend  the  time  and  effort  spend  on  the  reflection.  The  on-line  communication  allows  individuals  that 
arc  reserved  to  express  ideas  and  students  to  take  risk  more  than  hc/shc  might  in  class. 

The  students  submit  the  weekly  forms  and  at  the  end  of  the  term  submit  a three  questions  about  the  use  of 
technology  in  the  course.  In  the  future  we  will  have  students  write  a summativc  reflective  paper  in  which  the 
student  has  the  opportunity  to  review  the  weekly  reflective  journals  and  summarize  their  progress  during  the  term 
and  set  goals  for  the  future. 

In  Class  On-line  Assignments 

Throughout  the  semester,  the  students  develop  a list  of  goals  and  objectives  for  what  is  taught  in  social 
studies  at  each  grade  level.  The  students  use  the  National  Council  for  the  Social  Studies  Standards  (1994),  the 
Missouri  Show  Me  Standards,  the  Columbia  School  guidelines,  college  and  elementary  school  textbooks  to  research 
what  is  included  in  the  elementary  social  studies  curriculum.  These  documents  and  others  arc  used  to  develop  the  list 
of  grade  level  guidelines.  As  the  term  progresses,  the  students  use  the  list  of  goals  and  objectives  for  developing 
lessons  and  units. 
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In  the  second  week  of  ihc  semester,  the  students  form  a committee  of  four  to  research  the  scope  and 
sequence  for  social  studies  at  a specific  grade  level.  During  a class  period,  the  groups  develop  the  initial  list  of  ideas 
and  submit  their  ideas  to  the  listserv.  The  list  of  ideas  is  the  brainstorming  of  concepts  and  skills  at  a grade  level. 

The  students  reflect  and  react  to  the  list  throughout  the  semester  and  the  listserv  message  for  a grade  level 
is  revised  and  updated  as  new  ideas  arc  discussed  or  presented.  The  list  of  goals  becomes  more  defined  and  specific 
as  the  students  learn  new  concepts. 

By  having  technology  integrated  into  this  assignment  the  students  can  research,  write,  and  relied  on  the 
grade  level  objectives  throughout  the  term.  It  is  an  opportunity  for  students  to  construct  new  knowledge  as  needed 
and  to  share  the  knowledge  in  printed  form  to  their  peers  as  soon  as  it  is  completed.  The  resources  for  the 
assignment  are  the  current  materials  that  they  will  need  to  use  as  teachers.  The  assignment  is  student  focused  and  the 
list  of  ideas  is  distributed  by  the  students  and  retrieved  as  needed  by  the  students.  The  value  of  this  assignment  is 
that  the  ideas  arc  generated  by  students. 

By  using  the  listserv  to  share  the  brainstorming  of  small  groups,  technology  allows  students  to  receive  the 
ideas  from  their  peers  in  a quick  and  efficient  manner.  Messages  in  a listserv  can  be  revised  and  new  ideas  can  be 
added  as  time  passes.  The  instructor  doesn’t  have  to  develop  a new  handout  or  present  new  information.  It  is 
exciting  when  learning  about  a topic  can  be  expanded  over  time  and  new  messages  submitted  to  the  listserv  can 
include  thoughtful  and  creative  reflections  and  reactions.  A copy  of  work  during  class  can  be  provided  to  each 
student  instead  of  being  left  on  chart  paper  or  the  chalk  board  to  be  destroyed. 

I continue  to  look  for  methods  to  use  technology  during  class  time  for  share  information  during  small  group 
work.  1 am  considering  using  technology  to  brainstorm  a topic,  discussion  an  issue  or  peer  reviews  of  papers.  It  is 
exciting  to  consider  the  options  of  how  students  can  be  given  a task  on  the  listserv,  work  in  small  groups  with 
materials  on  reserve  in  a resource  center,  and  disseminate  the  information.  The  instructor  becomes  the  facilitator. 
This  is  a small  scale  example  of  distant  learning  in  a traditional  setting. 

Technology  in  Elementary  Schools 

Preservice  teachers  encounter  many  challenges  when  technology  based  instruction  is  employed  the 
classroom.  How  the  preservicc  teacher  responds  to  these  obstacles  determines  if  technology  is  an  integral  part  of 
classroom  instruction. 

A major  concern,  for  the  preservicc  teacher,  is  the  existence  of  a positive  technological  culture  in  the 
school.  For  successful  technology  based  instruction  in  schools,  a positive  (technological)  culture  in  the  school 
community  (Lee  1996)  needs  to  exist.  Intensive  technological  inservice  programs  and  sufficient  technical  support 
is  needed  to  provide  technology  based  instruction.  Teaching  in  a supportive  technological  environment  is  important 
but  basic  educational  needs  must  be  addressed  in  the  formation  of  a positive  technological  school  culture. 

How  many  computers  arc  available  for  classroom  instruction  is  important  to  the  preservicc  teacher.  The 
preservicc  teacher  must  know'  how  many  computers  arc  available  for  instruction.  Will  classroom  technological 
instruction  be  possible  or  will  planned  excursions  to  a computer  lah  be  needed? 

Another  challenge  for  preservice  teachers  is  knowing  what  technology  is  available  for  classroom 
instruction.  Arc  CD-ROMS  available  at  the  school’s  media  center?  Docs  the  school  possess  the  capability  for 
internal  school  communication  and,  thus,  increase  professional  dialogue  among  teachers  (Buckley  1995)  and 
students  (e-mail)?  Is  the  school  connected  to  the  World  Wide  Web  (Internet)? 

Developing  a community  of  learners  in  a teacher  education  classroom  is  key  to  modeling  how  the 
preservicc  student  will  develop  a community  of  learners  in  elementary  schools  (Bell  1995).  Preservice  teachers  need 
to  know  and  increase  their  own  technological  knowledge.  To  this  end,  docs  the  school  districl/huilding  provide 
inscrvicc  training  related  to  technological  classroom  instruction?  Is  adequate  technical  support  provided  by  the 
school  to  solve  dilemmas  which  develop  in  the  transmission  of  technological  based  instruction?  If  the  Internet  “goes 
down”  during  an  instructional  unit,  who  will  place  the  lesson  “on-line”?  Though  challenges  do  exist  in  the 
implementation  of  technological  based  instruction,  a developed  plan  of  attack  should  lessen  the  anxiety  of  instituting 
a curriculum  based  on  technological  instruction. 

In  developing  a lesson  plan,  preservicc  teachers  arc  introduced  to  many  planning  strategies.  A commonly 
used  planning  scheme  starts  with  searching  available  databases  with  search  engines.  Numerous  commercially  based 
search  engines  are  available.  Yahoo,  Alta  Vista  and  Lycos  arc  three  of  many  search  engines  used  in  planning. 
Information  garnered  by  the  search  engines  is  then  utilized  to  search  the  World  Wide  Web  (Internet).  The  Internet 
provides  the  preservicc  teacher  a multitude  of  information  sites,  many  of  which  possess  interactive  components,  that 
arc  applicable  to  classroom  instruction.  Through  class  iistscrvs,  e-mail  and  newsgroups  preservicc  teachers  share 
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their  Internet  discoveries  with  classmates.  This  promotes  professional  discourse  among  the  preservice  teachers. 
Technology  available  for  the  presentation  of  information  takes  many  forms.  PowerPoint,  photoshop  and  pagemakcr 
arc  a few  of  the  vehicles  employed  to  bring  technologically  based  instruction  to  the  classroom.  Coupled  with 
informational  gathering  knowledge,  preservice  teachers  arc  exposed  to  the  many  computer-based  methods  of 
leaching. 

Simulations  and  games  arc  frequently  used  methods  of  instruction  in  teaching  social  studies.  Simulations 
allow  students  to  engage  in  activities  that  would  otherwise  he  too  expensive,  dangerous  or  impractical  to  conduct,  in 
the  classroom  (Person  1996).  An  example  of  a social  studies  simulation  study  is  Rivcriown  Simulation.  This 
simulation  is  concerned  with  the  rebuilding  of  a city  located  on  a river 

(www.cmich.edu/public/gco/rtowninlro.htinl).  As  mentioned  earlier,  use  of  the  Internet  is  a popular  technological 
tool  in  the  classroom.  In  the  technological  world  of  the  Internet,  this  tool  allows  for  virtual  exploration  of  numerous 
topics.  The  Internet  empowers  the  classroom  teacher  to  transport  the  learner  to  the  far  reaches  of  the  world.  Instant 
visual  access  with  the  Internet  combined  with  instant  written  communication  abilities  provides  the  student  and  the 
preservice  teacher  an  endless  opportunity  of  knowledge  constructing  possibilities. 

I wish  my  teachers  in  clem,  school  has  used  integrated  themes  like  we  are  writing  -social  studies  would 

have  been  a lot  more  fun!  I used  to  hale  it  and  am  learning  to  like  it  again. 

1 guess  when  I think  about  student  leaching,  1 wonder  if  1 can  do  all  the  work  that  goes  into  a unit  and  be 

leaching  at  the  same  lime. 

Conclusions 

Research  continues  to  be  lacking  on  how  to  use  technology  as  a tool  for  teaching  social  studies  (Benson 
1996).  Instructors  and  preservice  students  are  challenged  to  develop  methods  for  incorporating  technology  that 
facilitates  a community  of  thinkers.  As  instructors  and  preservice  students  make  the  use  of  U 'hnology  an  integrated 
component  of  their  daily  life  and  are  proficient  user  of  technology,  then  educators  can  discover  new  methods  for 
combining  thinking  and  technology.  To  use  higher  level  thinking  skills  and  to  use  technology  lake  time  and  require 
advanced  skills,  educators  must  consider  if  the  investment  is  worth  the  return. 

Bach  time  technology  is  integrated  into  the  course  there  arc  new  challenges  and  successes.  Remember  that 
the  instructor  is  not  the  only  teacher  in  the  classroom,  students  arc  excellent  in  providing  expertise  in  technology. 
Sometimes  the  simple  task  can  be  the  most  rewarding,  such  as  when  students  get  “empowered”  w>'cn  they  learn  to 
cut  and  paste  Ur  other  location  besides  word  processing  documents.  We  have  learned  that  it  is  important  to  preasscss 
the  students  skills  level  in  using  each  form  of  technology  and  adjust  our  instruction  on  the  use  of  technology 
accordingly.  We  recommend  the  instructor  select  one  clas'.  assignment  or  task  each  term  and  determine  if 
technology  can  be  used  to  enhance  the  quality  of  the  learning  process. 

The  shift  in  the  pedagogical  infusion  of  technology  is  key  to  encouraging  problem  solving  and  inquiry 
driven  approaches  (Bullcr  & Clouse  1994).  The  use  of  technology  in  the  course  begins  with  the  instructor  initiated 
ideas,  expectations  and  assignments.  As  the  semester  progressed  the  students  used  technology  to  form  a community 
of  learners  hy  establishing  common  goals,  working  in  groups,  sharing  resources,  and  collaborating  on  projects. 
When  students  become  vested  in  the  writing  and  using  technology  then  a community  of  thinkers  is  formed  and  all 
groups  do  not  reach  this  level  of  responsibility.  The  level  of  student  involvement  and  ownership  in  the  process  varies 
each  term. 

One  students  reflective  journal  at  the  end  of  the  fall  1996  term  summarized  the  techniques  used  in  the 
course  for  providing  preservice  social  studies  teachers  with  the  skills  to  use  technology  and  create  a community  of 
thinkers. 


“It  has  been  beneficial  to  me  and  I have  learned  many  things  including  how  to  research  and  put  together  a 
unit.  I felt  that  I learned  a lot  about  teaching  social  studies.  What  I did  not  learn,  I feel  I now  have  the 
background  knowledge  to  research  on  my  own.  I also  learned  much  about  technology  in  the  classroom  and 
I hope  that  1 have  the  resources  to  use  the  things  I learned  in  my  own  classroom.” 
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Analysis-Based  Message  Design: 
Rethinking  Screen  Design  Guidelines 
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Abstract 

Since  the  1960's  computer  interfaces  have  evolved  from  text' based  to  multimedia  formats,  resulting  in  an 
evolution  in  the  research  regarding  interface  design.  This  article  describes  the  evolution  of  research  issues  from 
text-based  interface  design  guidelines  to  more  complex  issues  including  media  selection , interface  design,  and  visual 
design.  This  research  is  then  integrated  into  the  Analysis-based  Message  Design  (AMD)  process.  The  AMD 
process  divides  the  inteiface  design  process  into  four  action  steps,  the  fourth  of  which  features  the  ABC's  R'  US 
design  principles. 

Introdiu 

S-  .c  design  of  computer  screens  was  first  addressed  in  the  1960’s,  the  types  of  features  used  in 
instructional  products  have  not  just  expanded,  they  have  exploded  with  new  possibilities  such  as  graphical  user 
interfaces,  hypermedia,  digital  video,  and  virtual  reality.  Similarly,  the  area  of  interface  design  research  has  also 
evolved  in  order  to  adjust  for  these  changes. 

The  purpose  of  interface  design  research  is  to  determine  which  factors  help  computer  screen  layouts  support 
learning  and  which  factors  may  detract  from  learning.  The  goal  of  this  paper  is  to  describe  the  genesis  and 
development  of  existing  interface  design  research  and  to  describe  a process  called  Analysis-based  Message  Design 
(AMD)  which  incorporates  this  research  throughout  its  four-step  process. 

Print-based  Design  Research 

Computer-assisted  instruction  (CAI)  began  in  the  1960’s  with  text-based  programs  such  as  the  Stanford 
CAI  project,  the  IBM  1500  CAI  system,  and  the  PLATO  project  (Sacttlcr  1990,  p.  456).  The  first  interface  design 
guidelines  for  the  growing  number  of  text-based  CAI  programs  were  based  upon  research  on  print-based  instruction, 
especially  M.  A.  Tinker's  Legibility  of  Print  ( 1963).  While  Tinker's  work  was  a compilation  of  legibility  studies  on 
various  print  topics  (see  Tabic  l),  his  findings  were  the  basis  of  a series  of  prescriptive  guidelines  for  computer  text 
(Reynolds,  1979;  Rambally  & Rambally,  1987;  Shires  & 01s/,ak,  1992). 


Table  1.  Summary  of  Tinker's  Legibility  Findings  and  Corresponding  Design  Guidelines 


Topic 

Tinker’s  Findings 

Design  Guidelines  ‘ . j 

Text  Style 

• Italic  type  was  found  less  legible  than  normal  text. 

• Bold  text  used  for  emphasis  did  not  decrease  legibility. 

• Lower-case  letters  were  more  legible  than  all-caps. 

• Avoid  italic  type. 

• Use  bold  text  for  emphasis. 

• Avoid  using  all-caps. 

Text  Font 

• Sans  serif  and  serif  fonts  were  found  equally  legible; 
however,  readers  preferred  serif  fonts. 

• Use  serif  fonts  for  large  bodies  of  text. 

Text  Si/.e 

• Very  small  and  very  largo  type  sizes  decreased  reading 
speed. 

• Use  1 8pt  text  for  headings  and  1 2pt  text  for 
body  text. 

Line  Spacing 

• Leading  had  an  important  effect  on  legibility.  An  average 
leading  of  2-4  points  proved  the  most  legible  for  most  font 
and  line  length  combinations. 

• Double  space  text  or  skip  a line  between 
paragraphs. 

Line  Length 

• Readers  pre fared  moderate  line  lengths  and  ample 
margins. 

• Leave  white  space  in  the  layout. 

Text  Blocks 

• Square  blocks  of  text  were  found  less  desirable  than  regular 
blocks  of  text. 

• Avoid  justified  text  blocks. 

Color 

Combinations 

• The  most  legible  color  combinations  were  those  with  the 
greatest  contrast  between  the  text  and  the  background,  such 
as  black  text  on  a white  background 

• Use  dark  text  on  a light  background  or  light 
text  on  a dark  background. 
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Current  Interface  Design  Research 

The  1980’s  brought  a new  face  to  computers,  the  graphical  user  interface  (GUI).  While  the  first  computers 
wit!,  GUI  interfaces  were  manufactured  by  Xerox,  they  did  not  have  national  attention  until  Apple  announced  its 
line  of  GUI  computers  in  1984  (Cambcll-Kellcy  dz  Aspry,  1996).  This  new  interface  with  its  multimedia 
capabilities  presented  designers  with  a variety  of  text,  audio,  and  visual  design  options.  As  a result,  print-based 
guidelines  were  supplemented  by  interface  design  research  which  related  to  text,  as  well  as  multimedia. 

In  the  past,  some  designers  have  sought  prescriptive  design  guidelines,  lists  of  the  “do”s  and  “don't’s  of  interface 
design.  However,  as  Mcgarry  relates,  such  simplistic  lists  of  guidelines  cannot  adequately  address  the  design 
variables  in  complex  multimedia  products  (1991).  Therefore,  current  interface  design  research  involves  a rethinking 
of  the  concept  of  interface  design  guidelines.  From  the  1980’s  until  the  present  time,  the  focus  has  shifted  to  three 
main  types  of  interface  design  research:  media  selection  research,  interface  research,  and  visual  design  research. 

Media  Selection  Research 

Media  selection  research  is  based  upon  the  assumption  that  certain  media  arc  more  effective  than  others 
with  regard  to  certain  types  of  learners  or  content  (Kozma,  1991 ).  This  research  tests  the  effectiveness  of  different 
media  under  various  learner,  content,  and  delivery  environment  conditions.  Instructional  designers  can  use  this 
research  to  guide  their  selection  of  the  most  effective  media  features  to  meet  the  conditions  surrounding  ( heir 
instruction. 

Interface  Research 

Interface  research  deals  with  the  design  and  testing  of  human-computer  interfaces  (Apple,  1989;  Cates, 
1994;  Laurel,  1990;  Nielsen,  1990;  & Shncidcrman,  1992).  It  revolves  around  the  consistency  between  the  content, 
the  media,  the  functionality,  and  the  metaphoric  (heme  of  a multimedia  product.  Instructional  designers  can  use  this 
research  to  help  them  develop  a product  which  supports  clear  and  consistent  interaction  between  the  user  and  the 
computer  program. 

Visual  Design  Research 

Visual  design  research  addresses  the  visual  principles  used  in  interface  design.  These  heuristics  are 
extensions  of  the  ancient  Greek  aesthetic  principles  of  symmetry,  order,  emphasis,  unity,  and  balance.  These 
principles  have  been  extended  to  also  include  proximity,  and  parsimony  (Heinich,  Molcnda,  Russell,  & Smaldino, 
1996;  Williams,  1994;  Reilly  & Roach,  1986).  Instructional  designers  can  use  these  principle:  ''aesthetic  visual 

design  decrease  feelings  of  stress  among  learners  and  increase  feelings  of  confidence  and  stability  (Reilly  & Roach, 

! 986,  p.  39;  Rambally  & Rambally,  1 987,  p.  1 5 1 ). 

Analysis-Based  Message  Design  Process 

Computer  screen  message  design  addresses  the  three  layers  of  a screen:  the  content  layer  containing  the 
information  to  be  learned  in  the  form  of  various  media  features,  the  interface  layer  containing  the  means  through 
which  the  learner  interacts  with  the  informal  if  - presented  (text,  graphics,  etc.)  including  the  structure  through  which 
it  is  presented  (navigation,  metaphor,  etc.),  and  the  aesthetic  layer  which  organizes  the  information  and  interface 
according  to  aesthetic  principles.  The  AMD  process  reflects  the  way  in  which  these  three  layers  arc  designed.  It 
addresses  how  the  research  is  used  in  making  design  decisions.  AMD  is  a process  which  separates  the  design  phase 
of  the  Instructional  Systems  Design  (ISD)  model  into  a four-step  process.  During  this  process,  the  instructional 
design  team  will  (1)  identify  conditions,  (2)  select  appropriate  design  features,  (3)  resolve  conflicts  between  design 
features,  and  (4)  create  a design  layout  for  use  in  the  development  of  the  product  (sec  Figure  1 ). 

Step  1:  identify  Conditions 

The  first  step  of  AMD  does  not  directly  incorporate  interface  design  research.  Its  purpose  is  to  link  the 
analysis  and  design  phases  together  and  to  emphasize  the  importance  of  analysis  as  a foundation  for  design.  . While 
it  is  outside  the  scope  of  this  paper  to  discuss  how  to  col1  analysis  and  evaluation  data,  the  author  would  like  to 
stress  the  importance  of  this  data.  Therefore,  in  this  step  Hu  instructional  design  team  analyzes  the  conditions  which 
surround  the  instructional  context.  These  conditions  are  environmental,  learner,  or  instructional  needs  which  define 
or  create  boundaries  for  the  instructional  product.  While  collecting  or  compiling  the  conditions  which  result  from 
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Figure  J.  The  integration  of  AMD  into  a cyclical  design  cycle 

Renegotiation 
with  Client 


Identification  of 
Potential  Conflict 

analysis  or  formative  evaluation  data,  it  is  crucial  that  instructional  designers  identify  answers  for  the  following  four 
analysis  questions: 

• What  is  the  content  like? 

(subject  area,  type  of  learning,  scope  of  content,  expected  outcomes,  instructional  approach,  etc.) 

• What  are  the  learners  like? 

(age,  ability  level,  experience  level,  motivation,  culture,  learning  styles,  student  groupings,  etc.) 

• What  will  be  the  final  delivery  environment? 

(computer  type,  processor,  speed,  memory,  disk  space,  operating  system,  network  configuration,  color, 
sound,  and  video  capabilities,  etc.) 

• What  constraints  are  being  put  on  the  production  process? 

(lime,  cost,  resources,  etc.) 

It  should  be  noted  that  in  a cyclical  design  process,  evaluation  results,  not  just  analysis  results,  will  also 
generate  conditions  which  will  create  boundaries  for  the  product.  Sometimes  these  results  will  even  dc-cmphasizc 
the  importance  of  some  of  the  original  pre-design  analysis  results.  Therefore,  it  is  important  to  include  both  pre- 
design  analysis  results  and  formative  evaluation  results  since  both  of  these  types  of  data  affect  the  conditions 
surrounding  the  instructional  product. 

Step  2:  Select  Design  Features 

The  second  step  of  AMD  is  to  take  the  conditions  which  surround  the  instructional  context  and  to  select  the 
design  features  which  will  best  address  those  conditions.  In  the  case  of  each  condition,  there  are  some  features 
which  should  be  avoided  and  others  which  should  be  included  in  the  future. design  layout  (Romiszowski,  1981,  ch. 

' Gagne,  Briggs,  & Wagner,  1992,  ch.  11).  The  research  basis  for  these  decisions  should  stem  from  media 
selection  research  studies  and  subsequent  research  summaries  (Jonasscn,  1996;  Thompson,  Simonson,  & Hargrave, 
1996). 

The  most  important  issue  to  remember  during  this  step  of  the  AMD  process  is  that  methods  should  not  be 
chosen  arbitrarily.  On  the  contrary,  instructional  designers  should  use  instructional  and  learning  theories,  media 
selection  research,  and  personal  experience  to  support  design  feature  selection.  All  three  of  these  areas  arc 
important  sources  of  information.  While  some  instructional  designers  may  argue  that  professionals  in  the  field 
disregard  this  step,  the  author  would  argue  that  \s  not  the  case.  Professional  designers  have  a high  level  of  expertise 
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because  they  have  internalized  much  of  the  research  and  procedures  involved  in  the  design  process.  However,  when 
they  reflect  on  their  product  design,  it  becomes  evident  that  they  have  selected  features  which  were  tied  to  effective 
learning  of  the  content  while  taking  into  consideration  the  learner,  environmental,  and  process  conditions  which 
surround  the  instructional  context. 

Example: 

Let  us  imagine  that  there  is  an  instructional  designer  who  is  working  on  an  instructional  product  to  leach 
Spanish  conversational  skills.  In  the  first  step  of  the  AMD  process,  he  discovered  a number  of  conditions.  For  the 
sake  of  brevity,  let  us  concentrate  on  three  of  those  conditions,  the  content/ski  1 1 of  Spanish  conversation,  the  low 
motivation  level  of  the  learners  and  the  delivery  computer  configuration  (networked  486  PCs  with  SMB  of  RAM  but 
lacking  sound  cards  and  CD-ROMs).  During  the  second  step  of  the  AMD  process,  the  designer  will  determine 
which  design  features  will  be  most  effective  for  these  conditions  (see  Table  2). 


Table  2.  Selection  of  design  features  based  on  three  conditions  of  content , motivation,  and  delivery  computer 
specifications'  


Condition: 

Features  to  Include: 

Features  to  Avoid:  I 

Spanish  conversation 

• Text 

• Graphics  (to  support  text) 

• Digital  audio 

• Digital  video 

Low  motivation  level  of  the 
learners 

• High  interaction 

• Multimedia  features  (graphics,  audio, 
video,  etc.) 

• Learner  control 

• Real-world  examples  (graphics,  video, 
etc.) 

• Simulations 

• Drill  Structures 

• Fixed  linear  navigation  structure 

Networked  486  PCs  with 
4MB  of  RAM  but  lacking 
sound  cards  and  CD-ROMs 

• Speed  intensive  programming 

• Digital  audio 

• Digital  video 

Step  3:  Resolve  Conflicts 

The  third  step  of  AMD  resolves  the  conflicts  which  may  exist  between  the  features  to  be  avoided  and  the 
features  lo  be  included  for  all  the  conditions  surrounding  the  instruction.  Conflicts  occur  when  a feature  is  desirable 
(to  be  included)  with  regard  lo  one  or  more  conditions  and  undesirable  (to  be  avoided)  with  regard  lo  another 
condition.  There  arc  four  ways  to  resolve  these  conflicts: 

• Avoid  the  feature. 

A conflicting  feature  which  is  not  expected  to  significantly  impact  the  ciicctivcncss  of  the  instruction  can 
simply  he  avoided. 

• Include  or  avoid  the  feature  according  to  the  condition  with  the  highest  priority. 

When  feature  selection  varies  with  respect  to  different  conditions,  however  one  condition  has  a high-impact 
upon  the  product  or  the  production  cycle  (such  as  instructional  objectives  or  cost  issues),  the  feature  can  be 
avoided  or  included  with  regard  to  the  highest  priority  condition.  The  focus  here  is  on  maximizing  the 
effectiveness  of  the  instructional  product  within  the  constraints  which  surround  its  development. 

• Include  the  feature,  but  address  its  effectiveness  during  formative  evaluation. 

If  it  is  uncertain  whether  a feature  will  improve  instruction  due  lo  the  conditions  which  surround  it,  the 
feature  can  be  included  in  the  design  and  while  being  flagged  for  formative  evaluation.  In  this  ease,  the 
risk  of  including  the  feature  can  he  minimized  by  testing  its  effectiveness  during  the  formative  evaluation 
phase. 

• Renegotiate  with  the  client. 

Although  it  is  not  common,  there  arc  times  when  a conflict  should  be  resolved  through  renegotiation  with 
the  client.  This  is  the  ease  when  including  or  avoiding  a given  feature  would  compromise  the  effectiveness 
of  the  instructional  product. 
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Example: 

Fealures  which  are  in  possible  conflict  need  special  consideration.  They  arc  too  important  to  simply  be 
avoided  since  they  meet  certain  conditions  of  the  instructional  environment.  Sometimes  the  solution  of  these 
conflicts  involves  more  than  one  of  the  four  resolution  strategics  listed  below.  For  example,  the  instructional 
designers  who  are  designing  the  Spanish  conversation  product  might  decide  to  forego  the  aspects  of  multimedia 
since  the  delivery  computers  do  not  have  the  hardware  requirements  for  such  features.  The  addition  of  digital  video 
would  probably  not  have  a major  impact  on  the  learner’s  auditory  practice  so  it  could  be  avoided  without 
compromising  the  instructional  effectiveness  of  the  product.  By  avoiding  digital  video,  the  program  would  take  up 
less  storage  space.  However,  it  is  crucial  to  have  digital  audio  in  order  to  teach  the  oral  and  auditory  skills  needed  to 
converse  in  Spanish.  In  this  case,  there  needs  to  be  a renegotiation  with  the  client  since  delivery  conditions  are  in 
direct  conflict  with  instructional  effectiveness  and  there  is  no  other  way  to  resolve  this  conflict.  In  the  end,  the  client 
agrees  to  invest  in  sound  cards  for  the  machines  but  not  CD-ROMs.  With  the  sound  cards  and  the  available  hard 
disk  space  on  the  machines  it  is  now  possible  to  have  digital  audio.  In  this  case,  conflicts  were  avoided  by  using  a 
combination  of  avoidance  and  renegotiation. 

Step  4:  Create  the  Design  Layout 

The  fourth  step  of  AMD  involves  the  creation  of  the  actual  design  layout.  This  layout  takes  the  design 
features  which  are  deemed  effective  for  the  conditions  that  surround  the  instructional  product  and  gives  them  a form 
for  the  screen.  Layout  creation  relies  upon  interface  research  and  visual  design  research.  The  results  of  both  of 
these  types  of  research  can  be  summed  up  with  seven  ABC’s  R’  US  design  principles:  alignment,  balance,  contrast, 
chunking,  repetition  (theme),  utility,  and  simplicity. 

• Alignment 

Alignment  refers  not  only  to  the  alignment  of  text,  but  also  to  the  alignment  of  all  visual  objects.  Objects 
have  six  points  of  alignment:  right,  center,  left,  top,  middle,  and  bottom.  The  eye  can  perceive  the  strongest 
alignment  when  objects  arc  aligned  on  the  outer  points  on  the  right,  left,  lop,  and  bottom. 

• Balance 

The  center  of  a page  or  a screen  is  its  fulcrum  of  balance.  Balance  is  achieved  when  the  weight  of  the 
objects  on  the  left  side  arc  relatively  equal  to  those  on  the  right  side  and  when  the  objects  on  the  top  half 
arc  relatively  equal  to  those  on  the  bottom  half.  On  a light  background,  objects  w'hich  arc  darker  have 
more  visual  weight  than  those  which  arc  lighter.  On  a dark  background  the  opposite  is  true. 

• Contrast 

Contrast  is  the  level  of  difference  in  appearance  between  objects.  Contrast  can  be  achieved  by  placing  a 
dark  object  by  a light  object  or  by  pulling  a thin-featured  object  by  a thick-featured  object.  If  an  object  has 
a high  level  of  contrast  from  its  surrounding  objects,  it  draws  attention  to  itself.  The  key  to  effective 
contrast  is  to  use  it  sparingly.  If  one  object  stands  out  on  a page  or  on  a screen,  it  will  draw  the  learner's 
attention.  If  a number  of  objects  contrast  on  a page  or  on  a screen,  they  can  distract  or  even  confuse  the 
learner. 

• Chunking 

Visual  chunking  refers  to  the  placement  of  objects.  Objects  which  carry  out  similar  functions,  such  as 
navigation  buttons,  should  be  chunked  or  placed  in  the  same  visual  zone  of  a page  or  a screen.  Likewise, 
text  headings  should  be  chunked  with  the  text  which  supports  them. 

• Repetition 

Repetition  has  many  related  layers.  It  refers  to  the  consistent  use  of  objects,  effects,  fonts,  sizes,  styles,  and 
colors  both  on  an  individual  page  or  screen  and  also  throughout  the  product  as  a whole.  Repetition  also 
refers  to  the  metaphor  or  theme  used  throughout  the  product.  In  this  sense,  all  objects  should  repeat  and 
support  the  same  visual  message. . 

• Utility 

Utility  refers  to  the  usability  of  the  visual  interface.  It  should  be  clear  to  the  learner  which  objects  arc 
interactive  and  where  the  learner  is  located  within  the  instructional  product. 

• Simplicity 

Simplicity  addresses  the  appropriateness  and  the  necessity  of  the  visual  objects  on  a page  or  on  a screen. 
The  purpose  of  visual  objects  is  their  instructional  value  nol  their  impact  value.  Objects  which  do  not 
support  instruction  can  detract  from  learning.  The  key  is  to  keep  the  visual  design  simple  and  only  include 
those  visual  objects  which  arc  necessary  to  convey  the  instructional  message. 


to  building  a usable  and  aesthetically  sound  design.  Let  us  see 
to  build  the  layout  design  for  the  praetiee  screens  of  the  Spanish 

ity  to  a Spanish  conversation  project 


When  building  layouts  for  the  various  types  of  screens  in  a given  interface,  the  principles  arc  used  in 
reverse.  Therefore  the  designers  use  simplicity  from  the  beginning  of  the  design  process,  since  the  key  to  effective 
design  is  focusing  on  instruction  and  ease  of  interaction  (see  Figure  2).  They  select  the  key  media  features  for  the 
interface  from  steps  two  and  three  which  are  most  integral  to  their  goal  of  effective  instruction.  In  the  ease  of  the 
Spanish  conversation  project,  the  key  features  are  conversation  text,  a graphic  to  support  the  conversation  text,  a 
digital  audio  recording  of  the  conversation  text,  a means  of  recording  learner  practice  conversations,  and  separation 
of  the  content  into  separate  topic  modules  (for  future  sample  screens,  the  topic  is  “charlando  (small-talk)”). 


Example: 

The  ABC’s  R’  US  principles  arc  the  keys 
how  the  instructional  designers  use  these  principles 
conversation  product. 

Figure  2.  The  application  of  the  principle  of  si  mplic 

Key  Features 

• Conversation  in  Text  Form 

• Graphic  to  Support  Conversation  Text 

• Conversation  in  Audio  Form 

• Means  of  Recording  Learner  Practice 

• Separation  of  Content  into  Topic  Modules 


Figure  3.  The  application  of  the  principle  of  utility  to  a Spanish  conversation  project 


Charlando 


Jose  : eDe  donde  eres? 

Carmen  Soy  de  Espana,  iy  tu? 

Jose  Soy  de  la  Argentina 
Carmen  i Qu6  fantastico! 


V 


1 of  4 Exit 

Listen 


* 4 


Record 


Next  the  designers  focus  on  utility.  They  decide  that  each  screen  needs  to  have  the  following  elements:  a 
title,  the  text  of  the  conversation,  a simple  graphic  image  to  focus  the  learner  on  the  topic  of  the  conversation,  a 
means  of  accessing  the  digital  audio  clip  of  the  conversation,  a means  of  recording  the  learner’s  practice 
conversation,  a means  of  moving  from  one  card  to  another,  and  a means  of  leaving  the  practice  area  of  the  program 
(see  Figure  3).  The  designers  decide  to  use  a short  tutorial  on  how  to  use  the  program  at  the  beginning  so  that  the 
learner  knows  how  to  interact  with  these  elements  and  their  is  no  need  to  repeat  instructions  for  using  the  interface 
on  each  of  the  module  screens. 


As  Ihey  apply  the  principle  of  repetition,  the  designers  decide  to  use  a Spanish  theme  to  echo  the  objective 
of  teaching  Spanish.  The  background  image  on  the  screen  uses  colors  which  are  present  in  the  Hags  and  crafts  of 
many  Spanish-speaking  countries.  The  pattern  is  based  upon  the  pattern  used  in  goods  woven  in  several  Central 
American  countries.  The  screen  reference  and  the  words  on  the  listen,  tape,  and  exit  buttons  arc  in  Spanish,  as  well, 
to  repeat  the  Spanish  theme.  These  thematic  elements  are  repeated  on  all  the  module  screens  to  create  a Spanish 
look  and  feel  to  accompany  the  Spanish  content  (sec  Figure  4). 

to  a Spanish  conversation  project 


Next  the  chunking  of  the  interface  elements  separates  like  elements  into  the  same  visual  zone.  The  zigzag 
red  patterns  also  serve  to  break  the  screen  into  three  different  visual  zones  so  that  the  various  types  of  elements  can 
be  chunked  separately.  The  top  zone  is  used  for  the  title  and  the  page  number.  The  middle  zone  holds  the  content  of 
the  screen  (the  text  and  accompanying  graphic).  The  bottom  zone  is  contains  the  navigation  and  interaction  buttons 
(see  Figure  5). 

Figure  6.  The  application  of  the ' principle  of  contrast  to  a Spanish  conversation  project 


After  the  elements  arc  chunked  into  the  different  visual  zones,  the  text  and  graphics  in  the  lop  and  bottom 
zones  is  no  longer  legible  (see  Figure  5).  Therefore,  the  designers  increases  the  contrast  by  using  black  text  and 
light  background  colors  to  maximize  the  contrast  and  the  legibility  of  the  text  in  the  heading,  and  the  bottom  buttons. 
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The  light  color  also  goes  with  the  repeated  Spanish  theme  which  was  earlier  established.  In  addition,  a heavier  font 
is  used  for  all  text,  including  the  conversation  dialogue  text,  to  increase  legibility  (sec  Figure  6). 


The  interface  appears  balanced  for  the  most  part;  however,  the  designers  notice  that  the  top  /.one  needs 
some  adjustment.  They  decrease  the  size  of  the  module  screen  number  so  that  it  increases  the  lefl/righl  balance  of 
the  layout  (sec  Figure  7). 


Finally,  the  designers  align  the  outer  edges  of  the  text  and  the  graphic  with  the  outer  edges  of  the  arrow 
navigation  buttons.  All  of  the  buttons  are  middle  aligned,  as  are  the  text  and  graphic  of  the  middle  zone.  The 
information  top  zone  is  bottom  aligned  to  align  it  parallel  to  the  red  design  (sec  Figure  8). 


After  working  through  the  seven  principles  of  the  ABCs  R’  US.  the  finished  design  (see  Figure  9)  is  now 
ready  to  enter  the  development  stage  where  it  will  be  the  basis  of  the  grid  layout  for  the  module  screens. 
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Conclusions 

The  changes  in  computer  technology  has  moved  the  realm  of  interlace  design  research  from  print-based 
guidelines  to  generative  interface  design  research  on  media  selection,  interface  design,  and  visual  design.  These 
three  research  types  arc  integrated  into  the  AMD  process.  The  true  strength  of  the  AMD  process  lies  not  so  much  in 
its  four-step  process,  but  in  its  integration  of  these  four  design  steps  with  current  types  of  interface  design  research. 
The  process  tools  of  conflict  resolution  and  the  ABC’s  R’  US  design  principles  provide  designers  with  simple  ways 
to  tackle  interface  design  decisions.  Although  one  example  of  the  AMD  process  is  included  in  this  paper,  future 
research  could  verify  if  using  the  AMD  process  and  the  principles  of  ABC’s  R’  US  increases  the  effectiveness  or 
efficiency  of  the  visual  design  process. 
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The  Affective  Domain:  A Model  of  Learner-Instruction  Interactions 


Roy  M.  Bohlin 

California  State  University,  Fresno 


Abstract 

As  instruction  is  now  being  designed  with  more  regard  to  affective  outcomes , the  importance  of  a model  to 
help  the  process  has  increased.  Krathwohl,  Bloom,  and  Masia  (1956  & 1964)  and  Gephart  and  Ingle  (1976)  laid 
the  groundwork  for  working  in  the  affective  domain.  Keller  (1983)  proposed  a model  for  the  interaction  of  learners 
and  instruction  focusing  heavily  on  the  affective  domain.  Martin  and  Briggs  ( 1986)  later  published  a landmark 
book  to  help  those  in  the  field  of  instructional  Technology  with  the  process  of  integrating  the  affective  and  cognitive 
domains.  Using  these  sources  as  a starting  point  a more  comprehensive  model  for  instruction  in  the  affective 
domain  was  synthesized.  This  paper  therefore  presents  a model  for  those  interested  in  the  design  and/or  research  of 
instruction  in  the  affective  domain.  This  model  is  an  integration  of  current  theories  and  models  in  the  affective 
domain.  It  is  a broad  and  comprehensive  model  which  represents  the  hierarchical  structures  and  interactions  of 
affective  and  related  cognitive  factors,  including  attitudes , beliefs,  values , anxiety,  motivation,  attributions, 
confidence,  and  interests. 

Theoretical  Framework 

Krathwohl,  Bloom,  and  Masia  (1956)  developed  one  of  the  first  taxonomies  of  educational  objectives  for 
the  affective  domain  (sec  Figure  1).  Krathwohl,  Bloom,  and  Masia  (1964)  then  added  such  affective  constructs  as 
altitudes,  appreciation,  and  valuing  within  the  hierarchy  for  the  affective  domain  (sec  Figure  2).  This  enhanced 
taxonomy,  however,  had  many  gaps  and  did  not  contain  many  other  important  affective  domain  constructs. 

Figure  /.  Krathwohl  et  al.'s  Taxonomy  of  the  Affective  Domain 

(Adapted  from:  Krathwohl,  D.,  Bloom,  B.,  & Masia,  B.  (1956).  Taxonomy  of  educational  objectives. 
Handbook  II:  Affective  domain.  New  York:  David  McKay.) 


Level 

1.0  Receiving 


2.0  Responding 


3.0  Valuing 


4.0  Organisation 


5.0  Characleri/ation 


Definition 

Being  aware  of  or  attending  to 
something  in  the  environment. 

Sufficiently  involved  to  the 
subject  that  one  seeks  it  out. 

Showing  some  definite  involvement 
or  consistent  commitment. 

Integrating  a new  value  into  one's 
general  set  of  values,  giving  it 
dominance  among  one's  priorities. 

Acting  consistently  with  the  new 
value. 


Meaning 

I'll  read  an  article  about  proper 
professional  behavior,  hut  1 won't 
promise  that  I'll  like  it. 

I agrc„  to  attend  several  optional 
faculty  meetings. 

I attend  and  participate  in  all 
faculty  meetings. 

I volunteer  to  lake  a leadership  role 
in  the  the  faculty  government. 

I help  to  restructure  the  faculty  govern- 
ance to  improve  its  effectiveness. 
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Figure  2.  Range  of  constructs  in  Krathwohl  et.  ai  \v  Taxonomy  continuum. 

(Adapted  from:  Krathwohl,  D.,  Bloom,  B.,  & Masia,  B.  (1964).  Taxonomy  of  educational  objectives. 
Handbook  II:  Affective  domain.  New  York:  Longman.) 
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Gephart  and  Ingle  (1976)  proposed  a broader  descriptive  taxonomy  of  the  affective  domain  (see  Figure  3). 
While  this  model  encompassed  more  of  the  affective  domain,  it  still  omitted  several  concepts  usually  placed  within 
or  linked  to  constructs  in  the  affective  domain  (such  as  motivation,  confidence,  and  attributions)  and  is  limited  in  use 
for  researchers  and  designers  of  instruction. 


Figure  Taxonomy  of  the  Affective  Domain  by  Gephart  and  Ingle 

(Adapted  from  Geph  'rt,  W.  J.  & Ingle,  R.  B.  (1976).  Evaluation  and  the  affective  domain.  Proceedings 
of  the  National  Symposium  for  Professors  of  Educational  Research.  Phoenix:  ERIC) 
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Keller  (1983)  presented  a macro-model  for  the  relationship  of  motivation,  performance  and  instructional 
influence.  Keller  (1983)  also  developed  a mode!  for  the  design  of  motivating  instruction.  This  model  was  later 
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named  ARCS  — an  acronym  for  its  four  categories  of  attention,  relevance,  confidence,  and  satisfaction.  Keller 
integrated  the  works  of  Bandura  (1977)  in  self-efficacy,  Weiner  (1979)  in  attribution  theory,  McClelland  ( 1976)  in 
motives,  and  a number  of  other  theorists  into  this  model. 

Building  on  the  body  of  literature  regarding  the  Affective  Domain,  Martin  and  Briggs  (1986)  published  an 
award-winning  book  on  integrating  the  affective  and  cognitive  domains.  The  authors  presented  an  instructional 
design  model  using  audit  trails  for  integrating  the  two  domains.  Later,  Bohlin,  Milhcim,  and  Viechnicki  (1993) 
integrating  factor  analyses  of  data  from  the  motivational  needs  of  learners  with  the  macro  and  ARCS  models 
presented  by  Keller,  proposed  a prescriptive  instructional  design  model  for  motivating  instruction-learner 
interaction.  While  this  model  had  some  promise  for  designers,  it  did  not  entirely  provide  for  the  needed  more 
comprehensive  coverage  in  the  affective  domain. 

In  addition,  many  other  researchers  outside  of  educational  settings  have  provided  some  insights  into  how 
affective  constructs  might  influence  learners.  Spiclbcrgcr  (1966)  and  Tobias  (1979),  for  example,  have  performed 
significant  work  in  the  area  of  anxiety. 

These  previous  models  and  theory  structures  arc  fairly  comprehensive,  however  they  individually  lack  the 
scope  of  the  enough  of  the  affective  domain  to  help  designers  of  instruction  and  research.  There  is  a need,  therefore, 
to  synthesize  and  integrate  a new  more  comprehensive  instruction-learner  interaction  model  for  the  affective 
domain.  The  importance  of  such  a comprehensive  model  is  that  it  would  provides  designers  and  researchers  with  a 
means  for  considering  a greater  number  of  affective  variables  when  planning  the  research  and  design  of  instruction. 
Such  a model  also,  hopefully,  would  provide  a starting  point  for  discussions  regarding  the  relationships  of  various 
affective  and  linked  cognitive  constructs,  as  well  as  providing  a framework  by  which  instruction  and  research  in  the 
affective  domain  might  be  planned.  This  paper  is  an  attempt  to  begin  the  creation  of  such  a model. 

Constructs 

There  are  a number  of  variables  important  to  those  who  work  in  the  affective  domain.  Following  arc 
several  definitions  of  some  of  the  constructs  that  arc  in  the  Model  of  Learner-Instruction  Interactions  in  the 
Affective  Domain. 

1 . anxiety  - an  emotion  described  as  the  degree  of  fear  or  dread  toward  an  object  or  situation 

2.  arousal  - the  level  of  stimulation  being  experienced  by  an  individual  which  influences  ’he  level  attention  in 
that  situation  at  that  time 

3.  altitude  - internal  slate  or  learned  predisposition  toward  or  against  something  or  a set  of  things  that  have  a 
some  influence  over  an  individuals  choices 

4.  attributions  - sets  of  inferences  that  one  makes  about  the  causes  of  success  or  failure  related  to  ones 
behavior 

5.  beliefs  and  opinions  - a set  of  convictions  that  one  has  accepted  based  upon  ones  experiences  and 
interpretations  of  those  experiences 

6.  confidence  - the  state  of  positive  emotions  and  cognitions  related  to  ones  ability  to  successfully  complete 
an  expected  task 

7.  expectancy  of  success  - ones  perceived  likelihood  of  achieving  a positive  outcome  in  a given  situation 

8.  interests  - a set  of  preferences  or  tendencies  toward  some  idea  or  object  usually  related  to  a stimulation  of 
curiosity 

9.  motivational,  level  - the  degree  of  effort  and  the  direction  of  that  effort  as  to  experiences  or  goals  one 
approaches  or  avoids 

10.  motives  - learned  goal-directed  tendencies  which  arc  directed  by  drives 

1 1 . perceived  relevance  - the  degree  to  which  a given  situation  is  perceived  by  an  individual  as  correlating  with 
his  or  her  needs  and  motives 

12.  satisfaction  - a feeling  that  can  be  described  as  the  extent  to  which  the  outcomes  of  a situation  match  ones 
expectations  and  perceptions  of  equity 

13.  self-efficacy  - the  degree  of  conviction  that  one  has  the  ability  to  successfully  cope  and  complete  the 
behaviors  necessary  for  the  desired  outcomes  in  specific  situation 

14.  values  - groups  of  attitudes  organized  around  a central  idea  related  to  the  perceived  worth  of  that  idea 

The  Mode" 

The  Model  of  Learner-Instruction  Interactions  in  the  Affective  Domain  (sec  Figure  4)  shows  the  interactive 
relationship  of  instructional! y important  factors  (such  as  attributions,  confidence,  attitudes,  motivation,  and  values). 
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Constructs  which  interact  with  others  arc  clearly  shown  with  directional  arrows.  The  model  uses  the  following 
representations:  (a)  predominately  affective  variables  are  represented  by  rectangles;  (h)  predominately  cognitive  or 
behavioral  variables  arc  represented  by  ovals;  and  (c)  external  variables  arc  represented  by  boxes  with  rounded 
corners.  In  addition,  the  model  also  shows  those  constructs  which  arc  dependent  upon  others,  by  using  arrows  to 
show  the  direction  of  influence. 


Figure  4.  An  instruction-learner  interaction  model  J'or  the  affective  domain 


Applications 

There  are  several  ways  that  this  descriptive  model  can  be  used,  hirst  it  can  be  used  as  a model  for  the 
planning  of  instructional  interventions.  Let's  consider  an  instructional  designer  who  is  working  with  an  audience 
that  has  some  degree  of  debilitating  math  anxiety.  Us’ng  the  model  (see  Figure  4)  it  can  be  seen  that  anxiety  can  be 
affected  by: 

• cognitive  evaluations  related  to  ones  abilities  and  knosvlcdgc  in  mathematics  (for  example  “I  don't 
understand  how  algebra  works"  or  “I  have  never  understood  how  to  do  formal  proofs  in  geometry.”) 

• confidence  related  to  the  specific  instructional  strategies  (for  example  the  lack  of  visual  aids  for  a very 
visually  dependent  learner,  or  individual  work  for  a learner  who  needs  to  “discuss”  ideas  with  peers  before 
trying  them). 


• previous  outcomes  or  results  of  instructional  interactions  (for  example  "I  have  never  done  well  in  nlgchra'’ 

or  “I  failed  the  last  test  even  though  i studied  six  hours”. 

The  designer  cun  therefore  begin  to  examine  ways  to  assess  the  needs  of  these  learners  to  decide  (he  best  course  o[ 
action  for  intervening  in  the  debilitation  of  the  anxiety. 

Another  use  of  the  model  might  be  for  a researcher  investigating  affective  variables.  Research  .volving  the 
construct  of  learner  confidence,  for  example,  can  he  identified  to  have  effects  from  (sec  Figure  4)  anxiety, 
attributions,  self-efficacy,  the  specific  instructional  environment,  abilities,  skills,  and  knowledge.  This  model  docs 
not  make  any  attempt  to  predict  which  effects  arc  the  strongest,  for  those  are  determined  by  the  specific  individual 
case.  The  model  does  attempt  to  be  very  inclusive  so  that  many  potential  factors  can  be  considered. 

Third,  Some  constructs  are  directly  determined  by  others  (such  as  attitudes  by  beliefs  or  confidence  by 
attributions).  The  model,  therefore,  suggests  that  if  teachers  (for  example)  want  to  change  the  attitudes  of  students, 
then  they  should  work  on  the  underlying  beliefs  of  that  attitude.  For  example,  if  I want  to  change  a student's  attitude 
toward  using  computers,  then  l should  examine  the  beliefs  of  that  student  toward  the  use  of  computers  in  general 
and/or  related  specific  beliefs  (such  as  the  belief  about  the  uses  of  computers  or  beliefs  about  ones  efficacy  in  using 
computers). 

A fourth  way  that  this  model  can  be  used  is  to  examine  how  instruction  might  affect  specific  affective 
variables.  For  example,  let  us  consider  the  construct  of  anxiety.  Using  the  model,  we  can  see  that  instruction  does 
not  directly  link  to  anxiety,  but  it  docs  connect  through  several  indirect  paths.  One  such  path  is  through  performance 
and  the  related  feelings  of  the  performance.  Another  is  through  performance  and  cognitive  evaluation  of  ones 
improved  abilities  and  knowledge  (directly  or  through  changes  in  selFeflicacy  or  attributions).  A third  possibility  is 
by  providing  instructional  strategics  that  improve  confidence  levels  about. success  in  the  instruction. 

Limitations 

This  is  not  an  instructional  design  model.  It  does  not  have  an  entry  point  to  hegin  the  process  of  designing 
instructional  interventions.  In  order  to  design  instruction  using  this  model,  one  would  have  to  also  use  an  ID  model 
of  some  type. 

This  model  is  not  a prescriptive  model.  It  does  not  directly  provide  strategics  for  instructional  designers  or 
researchers.  As  a descriptive  interaction  model,  it  merely  describes  suggested  theoretical  relationships  among 
behaviors,  affective  constructs,  and  related  cognitive  factors.  Users  must  have  some  knowledge  of  the  nature  of 
these  variables  to  make  appropriate  inferences. 

Implications 

Much  of  the  research  into  the  affective  domain  has  been  rather  fragmented,  each  concentrating  on  one  or 
two  affective  constructs.  The  lack  of  a global  vehicle  to  gain  insight  into  the  larger  picture  has  represented  a 
significant  deficiency  in  making  (he  affective  domain  accessible  to  practitioners  and  theorists.  This  paper  is  an 
attempt  to  begin  to  fill  that  gap,  by  providing  a model  that  can  help  provide  descriptive  information  on  affective 
constructs.  It  is  hoped  that  this  model  can  increase  the  knowledge  base  of  those  practitioners  and  theorists  interested 
in  instruction  and  research  in  the  affective  domain. 

Wc  now  have  the  opportunity  to  begin  directing  more  of  our  focus  and  efforts  into  applications  and 
research  within  this  important  arena.  Researchers  and  designers  can  help  to  contribute  more  data  to  this  model  in  a 
number  of  ways.  They  can  help  specifically  if  they: 

• validate  the  theoretical  connections  in  the  model  with  research  results; 

• add  proscriptions  for  specific  variables  based  upon  both  research  and  host  practice  experience;  and 

• provide  information  about  the  best  ID  models  for  integrating  inferences  made  from  the  use  of  this  model. 
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Abstract 

The  development  of  potentially  powerful  computer-based  tools  within  rich  learning  environments  is 
hampered  by  the  constraints  of  delivery  systems  (specifically  low-resolution,  low  - real  estate  displays).  In  designing 
a Web-based  tool  for  manipulating  time-based  information  the  authors  have  encountered  a lack  of  guidelines  or 
empirical  research  to  help  resolve  problems  related  to  displaying  timelines  on  72  dpi , 640x480  pixel  computer 
screens.  In  a study  comparing  linear  vs.  staggered  arrangement  of  standard  32x32  pixel  icons  on  an  interactive 
timeline  and  found  that  subjects  perform  significantly  faster  when  making  temporal  relationship  comparisons  with 
the  linear  arrangement  of  icons  on  a timeline. 

Rich  learning  environments  arc  often  envisioned  to  contain,  or  even  to  revolve  around,  powerful  computer- 
based  tools  (Allen  & Otto,  1996;  Barab,  Hay,  & Duffy,  in  press;  Edwards,  1995;  Land  & Hannatm,  1996;  Papal, 
1991;  Perkins,  1991).  In  the  face  of  technical  limitations  (specifically,  low  resolution  displays  and  limited  screen 
space)  that  arc  prevalent  in  educational  sellings  (President’s  Committee  of  Advisors  on  Science  and  Technology, 
1997)  the  visual  design  problems  entailed  in  actually  implementing  such  tools  are  serious.  Interface  designers  for 
computer-based  learning  tools  need  to  be  able  to  turn  to  guidelines  and  empirical  research  relevant  to  the 
representation  of  complex  information  in  relatively  low  resolution  environments  for  help  in  solving  those  problems. 

Timeline  displays  in  relatively  low-resolution,  low-real  estate  conditions 

The  authors  of  this  study  arc  currently  involved  in  the  design  of  a Wcb-bascd  learning  tool  in  which  the 
display  of  temporal,  or  time-based,  information  is  the  primary  vehicle  for  interaction.  The  tool  offers  multiple  views 
of  any  single  data  set  at  varying  degrees  of  detail: 

a)  a high  level  overview  in  which  only  clusters  of  data  points  are  visible, 

b)  a mid-level  view  in  which  individual  data  points  appear  differentiated  only  by  their  duration  indicators 
(bars  drawn  along  a timeline), 

c)  a mini-view  showing  the  user's  current  position  in  cither  the  high  level  (a)  or  the  mid-level  (b)  view,  and 

d)  a detail  view  in  which  individual  data  points  are  represented  by  standard  screen  icons  (32x32  pixels) 
which  differ  according  to  the  predetermined  topical  category  into  which  the  data  point  falls.  Each  icon  in 
this  view  is  accompanied  by  a text  title  naming  the  data  point  and  by  a duration  bar  indicating  the  defined 
starting  and  stopping  points  in  time  for  that  item  (Figure  1.). 


HBHvega.2:  USSR  VeiL^Coaet  HaUfeyProte^ 

Figure  /.  Single  icon,  text  label  and  duration  bar 

The  detail  view  (d)  of  this  design  presents  particular  problems  in  formal  design;  namely,  the  accurate 
representation  of  a reasonable  number  of  data  points  in  a formal  that  best  supports  the  quick  and  accurate 
determination  of  temporal  relationships  between  events.  Given  a 72  dpi,  13"  screen  display  (640x480  pixels),  which 
is  the  slated  or  implied  lowest  common  denominator  in  numerous  commercial  design  publications  (Kristof  & Satran, 


1995;  Lopuck,  1996;  Lynch,  1994;  Niederst  & Freedman,  1996;  Waters,  1996),  the  limitation  on  the  number  of  data 
points  that  can  appear  in  a single  display  is  quite  severe.  The  practical  design  question  becomes  how  to  lit  as  many 
data  points  as  possible  on  the  screen  and  still  provide  a display  of  the  information  that  affords  efficient  visual 
processing  by  the  user? 

Problems  with  guidelines 

In  surveying  the  available  guidelines  for  screen  design  the  authors  encountered  problems  similar  to  those 
described  by  Lee  and  Boling  (1995).  Guidelines  are  frequently  aimed  at  single  dimensions  of  a display  (size  of  text, 
use  of  color,  placement  of  icons).  Complex  representations  of  information,  like  timelines,  require  the  integration  of 
multiple  elements,  all  of  them  influencing  all  the  others  by  virtue  of  the  way  they  arc  combined  . Single  dimension 
guidelines  are  of  little  use  in  solving  complex  design  problems. 

Guidelines  specifically  regarding  the  design  of  data  displays  like  charts,  graphs  and  diagrams  (including 
timelines)  do  not  offer  guidance  for  low  resolution  situations  (Kosslyn,  1994;  Fleming  & Levie,  1996),  or  simply 
state  that  high-resolution  is  required  for  the  effective  design  of  such  displays  (Tuftc,  1997).  While  Tuftc  may  be 
correct  in  the  pure  case  of  designing  optimal  displays,  the  fact  remains  that  developers  of  learning  tools  must  do  the 
best  possible  job  within  existing  constraints. 

Empirical  research  and  timeline  displays 

To  date  the  authors  have  found  no  empirical  research  specifically  studying  the  effective  design  of  timelines, 
for  print  or  for  screen.  Numerous  studies  do  exist,  however,  in  which  print  displays  have  been  used  to  investigate 
mental  imagery  related  to  temporal  information,  and  cognitive  processes  in  temporal  problem-solving.  This 
literature  suggests  that  alternative  representations  should  be  available  for  learners  depending  on  the  nature  of  the 
tasks  for  which  they  will  use  temporal  information  (Friedman,  1989,  1990),  that  learners  will  not  intuitively 
understand  displays  of  temporal  information  (Uka,  1962),  and  that  learners  of  different  ages  will  have  varying 
degrees  of  difficulty  using  such  displays  effectively  no  matter  how  well  they  arc  designed  (Friedman,  1992;  Levin, 
1992). 

Representing  time-based  information 

The  power  of  spatial  representations  of  lime  lies  in  the  simultaneous  comparison  of  items  in  order  to 
perceive  relationships  that  are  not  readily  discernible  through  experience  alone.  Every  spatial  representation  of*  time 
emphasizes  some  aspect  of  the  information  at  the  expense  of  another  (Friedman,  1990).  Timelines,  an  instance  of 
spatial  representations  of  time,  emphasize  duration  relationships,  of  which  thirteen  distinct  types  (Figure  2). may 
exist  between  any  two  given  lime  durations  Allen  & Kautz,  1985); 

• equals  (1), 

• bcforc/aflcr  (2,3), 

• mccts/mct  by  (4,5), 

• ovcrlaps/ovcrlappcd  by  (6,7), 

• starls/slarlcd  by  (8.9). 

• during/contains  (10,1 1),  and 

• finishes/finished  by  (1 2,13). 


equals  (1) 

— — i ---- 

before  / after  (2,3) 

Z — 

overlaps  / overlapped  by 
(6,7) 

— 

starts  / started  by  (8,9) 

— 

during/  contains  (10,11) 

— “ 

finishes  / finished  by 
(12,  13) 

Figure  2 Visual  representation  of  the  thirteen  temporal  relationship  types . 

Although  timelines  are  frequently  used  to  represent  temporal  information,  they  are  by  no  means  universal 
- nor  is  the  use  of  the  abstract  measures  of  lime  which  allow  such  representations  (Friedman,  1990).  Our  society  is 
permeated  with  abstract  measures  and  representations  of  lime  (and  other  forms  of  information),  but  we  do  not 
necessarily  make  effective  use  of  any  but  the  most  basic  and  common  displays  (Tuflc,  1983).  In  other  words,  there 
arc  no  "universal"  expectations  or  conventions  that  the  designers  of  an  interactive  timeline  display  may  rely  on  to  he 
confident  that  its  users  will  interpret  its  contents  correctly,  but  there  are  some  commonly  used  representations  to 
which  people  may  be  more  or  less  accustomed. 

Paper-based  timelines  are  often  printed  on  oversized  paper  (on  the  horizontal  dimension,  if  not  the  vertical), 
or  reproduced  at  a very  high  resolution  in  order  to  capture  ail  the  text  and  images  required  to  convey  multiple  events 
simultaneously.  When  the  size  or  the  resolution  of  a timeline  is  reduced  drastically,  our  hypothesis  is  that  the 
simultaneous  perception  of  events  may  be  compromised. 

Pragmatic  problems  with  screen  size,  resolution  and  timeline  displays  using  32x32  pixel 
icons 

At  72  dpi,  a 640x480  screen  display  includes  a total  of  307,200  pixels.  At  1024  pixels  per  standard  32x32 
pixel  icon,  300  icons  could  fit  on  the  screen  in  a tiled  arrangement:  providing  they  did  not  include  text  lilies.  In  fact, 
after  subtracting  interface  elements  (scroll  bars,  window  titles,  timeline  labels,  etc.)  at  approximately  40  pixels 
vertically  and  45  pixels  horizontally  (a  conservative  estimate  based  on  one  horizontal  and  one  vertical  scroll  bar.  a 
labeled  timeline,  and  a 30  pixel  vertical  space  for  category  labels,  butlons  and  other  interface  elements)  the 
remaining  595x440  display  can  only  hold  approximately  255  icons,  again  tiled.  Allowing  for  a 64-pixcl  title  per  icon 
(64x32  pixels  per  icon),  this  number  drops  to  127  tiled  icons  in  that  same  space.  Of  course,  since  the  placement  of 
the  icons  is  dependent  on  their  specific  relationship  to  the  timeline,  they  are  never  likely  to  be  perfectly  tiled  so  as  to 
lake  maximum  advantage  of  the  available  pixels.  The  effective  number  of  standard-size  icons  that  may  fit  on  our 
hypothetical  timeline  screen  is  therefore  some  number  fewer  (generally  distinctly  fewer)  than  127.  Given  the  several 
thousands  of  data  points  possible  in  a given  query  for  the  project  under  development,  it  was  a matter  of  some 
concern  to  the  authors  that  the  arrangement  of  those  icons  be  optimized  for  this  relatively  limited  display. 

Smaller  icons  and  other  alternative  displays 

It  is  perfectly  possible  to  envision  simple  solutions  to  the  pragmatic  problems  in  formal  design  for  timeline 
displays  in  which  the  64x32  pixel  icon-plus-label  constraint  is  removed;  for  example,  events  might  be  represented 
by  titles  and  duration  bars  only,  or  even  by  duration  bars  only,  at  a savings  of  as  much  as  half  the  vertical  space 
required  for  individual  items.  The  authors  pursued  experimentation  with  icon  displays,  however,  for  reasons  related 
to  discri minabili ty  and  user  preference. 
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Discriminability 

Searching  for  individual  items  in  a complex  display  can  be  made  more  effective  when  the  items  themselves 
exhibit  high  discriminability,  or  differences  that  dearly  distinguish  them  one  from  the  other  (Horton,  1994;  Wolfe, 
1996).  Pictorial  representations  (icons)  arc  more  discriminate  than  text  labels  alone  (Fleming  & Levie,  1993),  and 
certainly  more  discriminable  than  duration  bars  alone.  While  the  use  of  icons  on  a timeline  does  not  dictate  high 
discriminability  (if  an  author  uses  the  same  icon  for  every  event,  for  example),  the  ability  to  display  icons  on  the 
timeline  provides  the  possibility  for  representations  of  events  to  he  well-differentiated. 

Preference  for  icons 

The  user-collaborator  working  with  the  authors  currently  has  expressed  a strong  preference  for  icons 
accompanying  text  labels  on  the  timeline,  and  expects  that  students  using  the  tool  will  want  to  create  new  icons  and 
attach  them  to  specific  types  of  events.  In  the  experience  of  the  authors,  similar  preferences  are  frequently  expressed 
by  users  and  user-designers  of  screen  displays.  A few  early  icon  studies  in  which  preference  measures  were 
employed  have  revealed  the  same  preference  (Edigo  & Paltcrson,  1988;  Guastcllo,  Traut,  & Korienek,  1989)., 
although  these  findings  arc  by  no  means  universal. 

Linear  vs.  staggered  arrangement  of  icons  on  the  timeline  display 

Two  common  timeline  strategies  (whether  or  not  icons  are  used  to  represent  data  points)  represent  events  as 
perceptual  "’/ars,"  stretching  from  their  beginning  point  to  their  ending  point  in  a consistent  relationship  to  a series 
of  constam  units  marked  off  at  the  lop  or  the  bottom  of  the  display.  The  two  strategics  vary  only  in  the  spatial 
arrangement  of  the  bars  representing  events  (Figure  3.).  The  first  arrangement  places  each  event  successively  higher 
on  the  screen  display,  and  places  each  event  further  to  the  right  depending  on  when  that  event  begins  in  lime  (linear 
arrangement).  The  second  arrangement  conserves  screen  space  by  placing  events  arbitrarily  on  the  vertical 
dimension  at  any  time  that  there  is  room  available  for  them  (staggered  arrangement). 

item  e 


item  d 

itemc 


itemb 
item  a 

LINEAR  /ARRANGEMENT  OF  EVENTS  (DATA  FONTS) 


itemc 

item  b item  d item  e 

item  a 

STAGGERED  ARRANGEMENT  OF  EVENTS  (DATA  FONTS) 

Figure  3.  Linear  v.v.  staggered  arrangement  of  generic  event  bars  on  a timeline. 

Purpose  of  the  Study 

The  author*  conducted  two  studies  to  determine  whether  subjects  using  an  interactive  timeline  work  faster 
and  make  fewer  errors  in  judging  temporal  relationships  between  pairs  of  events  when  Ihe  events  on  the  timeline 
appear  in  a hnen,  arrangement  or  in  a staggered  arrangement.  The  first  study  also  examined  subject's  self-reported 
patterns  of  searching  for  items  on  the  timeline  displays,  and  their  subjective  impressions  of  the  difficulty  of  using  the 
timeline  display. 

A linear  arrangement  of  icons  on  a timeline  was  expected  to  result  in  more  scrolling  of  the  screen  display 
since  every  new  data  point  appears  above  the  previous  one  and  the  total  display  very  quickly  exceeds  the  screen 
space  available  without  scrolling.  A linear  display  was  also  expected  to  result  in  some  loss  of  simultaneity  since 
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events  that  arc  close  in  lime  might  not  appear  on  the  screen  together,  again  owing  to  the  steep  rise  created  by  placing 
each  data  point  higher  on  the  screen  than  the  previous  one  (Figure  4).  For  these  reasons  subjects  were  expected  to 
take  more  time  judging  temporal  relationships  using  the  linear  arrangement. 


events  close  in  time  that  do 
not  appear  in  the  display 
together  without  scrolling 


■ visible  portion  of  the  display 

Figure  4.  Linear  arrangement  of  data  points  with  loss  of  simultaneity  and  high  requirements  for  scrollin'. 


The  staggered  arrangement  requires  that  subjects  ignore  differences  on  the  vertical  dimension  in  order  to 
judge  temporal  relationships  between  events  correctly,  since  events  may  appear  cither  above  or  below  others 
regardless  of  whether  they  begin  earlier  or  later  on  the  lime  scale.  In  addition,  the  staggered  display  was  expected  to 
be  "noisier,"  with  more  icons  appearing  on  the  screen  at  one  time,  and  therefore  potentially  distracting  as  subjects 
tried  to  judge  relationships  between  events.  The  staggered  display  would  also  require  scrolling  for  even  moderately 
large  data  sets,  although  it  was  presumed  not  to  require  as  much  scrolling  as  the  linear  display.  Consequently,  the 
staggered  arrangement  was  expected  to  result  in  more  errors. 

Method 

Participants 

Forly-iwo  undergraduate  students  (19  men  and  23  women,  mean  age  = 19.4  years)  ai  a large  mid-western 
university  volunteered  to  participate.  All  participants  were  recruited  from  non-major  a class  in  tiic 
Telecommunications  department.  Volunteers  were  given  partial  credit  towards  a class  assignment  for  participation. 

Materials 

All  test  sessions  were  conducted  in  a computing  lah  run  hy  the  University.  The  lab  was  equipped  with  40 
computers,  each  running  Pentium  II  processors,  the  Microsoft  Windows  NT  operating  system,  and  64  megabytes  of 
RAM.  All  displays  were  color  SVGA,  measured  15  inches  diagonally,  and  were  set  to  800  x 600  resolution.  The 
software  used  to  run  the  data-col lection  instrument  was  HotJava  version  1.1. 

The  data-collcction  instrument  was  a software  program  written  in  Java  by  the  researchers.  The  instrument 
interface  included  several  important  features  (Figures  5-6). 

• A tiniclinc/icon  display  which  could  be  scrolled  through  both  vertically  and  horizontally. 

• Questions  to  the  participant. 

• Radio  buttons  providing  a constant  selection  of  answers  to  the  questions  (“Yes",  “No",  and  “I  can't 
telD. 

• A button  to  retrieve  the  next  question. 

In  addition,  the  instrument  displayed  several  specific  characteristics. 

• The  timcline/icon  display  consumed  approximately  30 c/e  of  the  total  display  area 

• Accompanying  every  icon  was  a thin  black  bar  that  stretched  from  the  event's  starling  point  to  its 
ending  pomt  in  accordance  with  the  timeline  below  it. 

• Clicking  on  the  thin  black  bar  associated  with  each  icon  caused  light  grey  lines  to  descend  from  (he 
starting  and  ending  points  down  to  the  timeline. 
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Figure  5.  Data  collection  instrument  interface  showing  the  linear  arrangement  of  icons. 


Procedure 

Testing  was  conducted  on  various  days  over  a two-wcck  period  between  9 a.m.  and  7 p.m.  Participants 
were  allowed  to  select  their  own  session  limes.  As  participants  arrived  at  the  lab  they  were  met  by  one  of  the 
researchers  and  given  instructions  to  review  the  study  information  sheet,  sit  down  at  an  available  computer  of  their 
choice,  and  log  into  the  network  using  their  university  identification.  All  members  of  the  research  team  used  a script 
for  the  duration  of  the  interaction  with  each  participant. 

From  a deck  of  50  shuffled  cards  (25  for  each  condition),  each  participant  was  given  one  card  which  had  an 
8-digit  password  printed  on  it  that  would  inform  the  computer  system  which  condition  to  present  to  the  participant. 
Participants  were  then  instructed  to  open  the  application  called  HotJava  and  enter  the  password  on  the  card  when 
prompted.  Next  the  participants  were  provided  a URL  to  enter  into  the  application  which  called  up  the  first  practice 
screen  for  the  instrument.  At  this  time  the  screen  was  maximized  to  consume  the  entire  display. 

Participants  were  next  given  a short  demonstration  of  the  basic  functionality  of  the  instrument.  The 
demonstration  included  navigating  the  timeline  display  using  the  scroll  bars,  activating  the  drop-down  “legs”,  and 
moving  on  to  the  next  question.  If  the  participant  did  not  have  questions  regarding  the  instrument,  they  were  then 
presented  with  a second  practice  screen  which  they  could  use  for  as  long  as  they  wanted  before  beginning  the  test. 
Once  the  test  was  initiated,  researchers  did  not  interfere  with  the  session. 

Subjects  answered  12  randomly  ordered  yes/no  questions  concerning  the  temporal  relationship  of  two 
events  on  the  timeline.  Six  temporal  relationships  (representing  all  the  core  relationships  without  their  direct 
opposites,  i.c.,  “contains”  but  not  “contained  by”)  were  addressed  by  the  questions,  with  two  questions  devoted  to 
each.  The  timeline  display  remained  exactly  as  the  subject  left  it  from  question  to  question,  so  subjects  frequently 
had  to  scroll  the  display  in  order  to  locate  the  events  named  in  the  question  before  they  could  answer  since  those 
events  were  not  necessarily  visible  on  the  screen.  The  same  twelve  questions  were  answered  by  all  subjects  in  both 
conditions. 
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The  instrument  recorded  three  types  of  data. 

• the  length  of  lime  in  seconds  from  the  appearance  oflhc  second  practice  screen  until  the  subject 
clicked  to  go  on  to  the  questions 

• the  subject’s  answer  (yes,  no,  I can’t  tell)  foi  each  question 

• the  length  of  lime  in  seconds  from  the  appearance  of  a new  question  until  the  subject  clicked  to  go  on 
to  the  next  question 

All  ihc  data  from  the  instrument  were  collected  and  stored  automatically  at  the  conclusion  of  each  subject’s 
session  in  a database  located  on  a separate  computer.  The  data  were  identified  only  by  tne  time  of  the  session  and 
by  the  8-digit  password. 
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Figure  6.  Data  collection  instrument  interface  showing  the  staggnred  arrangement  of  icons. 

Results 


Completion  time 

Completion  time  data  were  analyzed  using  a one-way  analysis  of  variance,  and  revealed  significant 
differences  between  the  linear  and  staggered  displays,  FC1.41)  = 8.14,  p<  .05.  Means  indicated  that  subjects  using 
the  linear  display  look  longer  to  complete  the  test  ( 1 1 82.86)  than  subjects  using  the  staggered  display  ( 1 151 .04). 

Answer  accuracy 

Answer  accuracy  data  were  divided  into  two  parts  for  analysis,  including  correctness  ("Yes'V'No")  and 
uncertainly  ("I  can’t  tell  ").  One-way  analyses  of  variance  were  conducted  for  both  measures.  A significant 
difference  was  found  for  correctness  F(l,  41)  = 4.52,  p < .05,  with  means  indicating  that  subjects  using  the  linear 
display  made  more  errors  (1.42)  than  subjects  using  the  staggered  display  (.57).  No  significant  difference  was  found 
between  conditions  for  the  uncertainly  measure. 
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Answer  accuracy  with  “overlaps/overlapped  by”  removed 

Post  hoc  analyses  revealed  that  removal  of  the  “ovcrlaps/overlapped  by”  temporal  relationship  from  the 
data  set  resulted  in  no  significant  differences  for  answer  accuracy  across  conditions.  Further  analysis  revealed  a 
significant  difference  for  correctness,  F (1, 41)  = 6.21,  p < .05,  across  conditions  for  the  “ovcrlaps/ovcrlapped  by” 
temporal  relationship.  Means  indicated  that  subjects  using  the  linear  display  made  more  errors  on  items  that 
represented  the  “ovcrlaps/overlapped  by”  relationship  (.52)  than  subjects  using  the  staggered  display  (.14). 


Discussion 

Completion  Time 

Results  for  completion  time  were  inconsistent  with  the  authors’  original  expectations.  Subjects  using  the 
staggered  display  look  approximately  six  minutes  longer  to  complete  the  lest  than  subjects  using  the  linear  display. 
The  authors  speculate  that  the  amount  of  scrolling  required  to  locate  target  events  on  the  timeline  using  the  linear 
arrangement  was  offset  by  the  fact  that  subjects  only  needed  to  scan  a single  column  of  icons  backward  and  forward 
in  order  to  locale  a target  event.  In  contrast,  subjects  using  the  staggered  arrangement  had  to  scan  entire  screens  of 
icons  with  no  regular  vortical  or  horizc  Mal  pattern.  They  also  had  to  control  the  scrolling  of  the  display  more 
precisely  to  ensure  tha*  they  had  scanned  ' /cry  icon. 

The  pairs  of  target  events  required  to  answer  questions  could  he  seen  on  the  timeline  display  at  the  same 
time  for  six  questions  in  the  linear  arrangement  and  only  three  questions  in  the  staggered  arrangement,  which  means 
that  the  subjects  using  the  staggered  arrangement  had  to  locate  one  event  and  then  the  other  twice  as  often  as 
subjects  using  the  linear  arrangement.  Subjects  using  the  staggered  arrangement  may  have  taken  more  time  both  in 
looking  at  the  display  and  in  scrolling  around  the  display  than  did  the  subjects  using  the  linear  arrangement. 

Once  subjects  using  the  staggered  display  located  target  events,  they  may  have  taken  more  time  to  identify 
the  temporal  relationship  between  events  because  the  overall  display  they  were  scanning  contained  more  visual 
noise  (icons  that  were  not  the  target  events)  than  did  the  linear  arrangement. 

Answer  Accuracy 

Initial  results  for  answer  accuracy  were  also  inconsistent  with  the  authors’  expectations.  Subjects  using  the 
linear  display  made  more  errors  than  subjects  using  the  staggered  display.  However,  post  hoc  analyses  revealed  that 
removal  of  the  “ovcrlaps/ovcrlapped  by”  temporal  relationship  from  the  data  set  resulted  in  no  significant 
differences  for  answer  accuracy  across  conditions.  The  authors  had  speculated  mat  visual  noise  in  the  staggered 
arrangement  would  interfere  with  subjects’  ability  to  make  accurate  identification  of  relationships  and  lead  to  more 
errors  in  that  condition. 

Answer  accuracy  with  “overlaps/overlapped  by”  removed 

Further  analysis  revealed  that  subjects  using  the  linear  arrangement  made  more  errors  identifying 
“ovcrlaps/ovcrlappcd  by”  relationships  than  did  subjects  using  the  staggered  display.  The  authors  arc  unable  to 
attribute  this  result  to  the  questions  themselves  or  to  the  nature  of  the  temporal  relationship,  and  unable  to  explain 
what  element  or  elements  of  the  displays  might  account  for  the  result. 

Limitations  of  the  study 

Temporal  relationships  between  events  are  easier  to  sec  in  diagram  form  than  to  dcscrihc  in  question  form. 
The  authors  recognize  that  more  rigorous  pretesting  of  the  stimulus  questions  is  required  to  ensure  that  the  questions 
arc  stated  as  clearly  as  possible. 

The  data  set  chosen  lor  the  study,  manned  and  unmanned  space  exploration  missions,  was  chosen  from 
several  candidates  for  which  there  was  ready  availability  many  defined  data  points.  The  authors  speculated  that  the 
content  of  the  data  set  might  be  perceived  as  scientific  and  therefore  somewhat  neutral,  or  even  mildly  appealing  to 
more  than  one  audience.  The  unanticipated  result  of  this  choice  was  that  almost  all  the  events  named  in  the  stimulus 
questions  were  numbered  (Venera  3,  Mars  5,  and  so  on).  Subjects  reported  incorporating  this  feature  of  the  data  set 
into  their  search  strategics  in  both  the  linear  and  the  staggered  anangement  conditions  by  searching  for  an  event  of 
the  same  name,  noting  its  number  and  then  scrolling  either  forward  or  backward  depending  on  whether  the  number 
of  the  target  item  was  higher  or  lower  than  the  one  found  . Although  the  study  will  have  to  be  duplicated  with  a 
different  data  set  to  determine  what,  if  any  effect,  this  choice  had  on  the  results  of  the  study,  the  authors  anticipate 
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thai  the  effect  may  apply  equally  to  both  conditions  since  subjects  in  both  conditions  reported  using  the  numbers  to 
search 

Although  the  logical  anticipation  of  the  authors  was  that  staggered  displays  might  afford  easier  comparison 
of  events  because  more  events  could  conceivably  appear  on  the  screen  at  one  time,  the  displays  created  by  the 
algorithm  in  the  instrument  did  not  conform  to  this  expectation.  Of  the  twelve  items,  six  in  the  linear  condition 
showed  both  target  events  on  screen  at  the  same  time  compared  to  only  three  in  the  staggered  condition.  The  authors 
recognize  that  this  outcome  may  have  affected  the  time  required  for  subjects  in  the  staggered  condition  to  search  for 
target  items.  Experimentation  will  have  to  be  done  with  multiple  data  sets  to  determine  whether  the  displays  for  this 
instrument  were  typical. 

Conclusion 

Despite  the  limitations  of  the  study,  the  difference  in  time  taken  between  subjects  in  the  linear  and 
staggered  timeline  conditions  strongly  suggests  that  linear  arrangements  may  be  a more  efficient  representation  for 
time-based  information  in  low-real  estate  displays  than  are  staggered  arrangements  with  respect  to  searching  for 
specific  events. 
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Abstract 

The  purpose  of  this  study  was  to  determine  the  effects  of  presentation  interference  on  students’  ability  to 
learn  from,  and  their  beliefs  regarding,  computer-generated  presentations.  Informed  by  single-element  screen 
design  studies  and  professional  practice,  this  combined-element  study  compared  a contextually  based,  intentionally 
interference  free  presentation  against  two  presentations  containing  multiple,  typical  design  flaws.  One  hundred  and 
eighteen  undergraduate  students  were  randomly  assigned  to  three  groups  and  subjects  individually  viewed  one  of 
the  three  treatments.  Following  the  presentation,  subjects  completed  a posttest  and  a belief  questionnaire. 

Three  hypotheses  were  tested:  subjects  receiving  presentations  with  no  intentional  presentation 
interference  would  perform  better  on  the  posttest;  subjects  receiving  presentations  with  less  interference  would 
perform  better  than  those  receiving  presentations  with  more  interference;  and  subjects  in  the  interference-free 
presentation  group  would  report  more  positive  beliefs  regarding  computer  generated  presentations  than  would 
subjects  receiving  presentations  with  interference. 

Results  indicated  significantly  higher  posttest  scores  for  the  interference  free  presentation  group  than  for 
either  of  the  two  groups  receiving  presentations  containing  interference.  Results  also  suggest  that  increasing 
intentional  interference  does  not  always  lead  to  increased  learning  cost.  In  some  contexts,  elements  that  can  be 
reasonably  expected  to  hinder  learning  may  have  positive  or  neutral  effects . Subjects  in  all  three  groups  agreed  that 
computer  generated  presentations  are  generally  beneficial,  however  responses  from  the  group  with  no  interference 
and  the  group  with  four  intentionally  introduced  types  of  interference  were  both  significantly  higher  than  responses 
from  the  group  with  two  intentionally  introduced  types  of  interference.  Although  the  well  designed  presentation  and 
a presentation  containing  intentional  interference  both  resulted  in  strong  beliefs  regarding  the  beneficial  nature  of 
computer  generated  presentations,  treatments  containing  presentation  interference  significantly  reduced  learning. 

Introduction 

The  advances  in  presentation  media  have  been  so  exciting  that  many  people  have  focused  on  the  media  and 
have  forgotten  to  examine  critically  the  purpose,  the  content,  and  the  overall  visual  appearance  of  products. 

Particularly  in  cases  where  there  is  no  instructor  to  clarify  meanings  and  relationships,  it  is  vital  that  the  mcssagc(s) 
of  a presentation  be  easily  accessible.  Poorly  designed  screens  have  the  potential  to  turn  learners’  attention  away 
from  the  message  and  onto  the  delivery  system  itself.  Although  many  design  guidelines  have  been  produced,  the 
limitations  of  most  standards  arc  that  (a)  assertions  are  not  fully  supported  by  research  , (b)  context  is  ignored,  and 
(e)  prescriptive  guidelines  arc  not  conducive  to  formative  and  summativc  evaluation  processes.  Research  is  needed 
that  combines  design  elements  in  realistic  ways  to  determine  factors  or  combinations  of  factors  that  hinder  learning 
while  acknowledging  the  context-driven  nature  of  successful  design  decisions.  Further,  developers  need  a method  of 
evaluation  and  feedback  that  can  be  used  to  improve  existing  presentations. 

Purpose  of  the  Study 

A review  of  screen  design  research  shows  that  most  studies  focus  on  the  effects  of  a single  clement  (as 
opposed  to  multiple  element)  change  on  subjects’  ability  to  learn  from  presented  material.  Although  studies  indicate 
that  single  element  changes  can  affect  learning,  most  existing  research  docs  not  address  the  difference  between  what 
was  learned  from  a realistic  presentation  and  what  could  have  been  learned  from  a rigorously  critiqued  and  revised 
presentation. 

The  present  study  used  a different  approach.  First,  an  “error  free"  visual  arrangement  for  a specific  content 
area  was  developed.  This  was  accomplished  hy  adapting  screen  design  prcscriptivcs  from  research  and  professional 
practice  to  a specific  presentation  topic.  Individual  screen  elements  (c.g.,  color,  text,  etc.)  were  handled  according  to 
established  guidelines  from  professional  practice  and  single  element  research,  insofar  as  results  from  such 
prcscriptivcs  were  appropriate  to  the  context.  Next,  the  presentation  was  critiqued  by  three  professional  presentation 
developers,  revised,  and  approved  as  “error  free,"  that  is,  no  distracting  Haws  were  detected.  Finally,  the  study 
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compared  the  error  free  version  against  two  additional  presentations  to  which  presentation  interference  had  been 
intentionally  added. 

Researchers  (e.g.,  Seels,  1994)  have  called  for  more  specific,  operationally  defined  constructs  that  can  be 
used  in  multiple  element  research.  The  term  “presentation  interference”  is  used  in  this  study  to  refer  to  distracting 
Haws  in  presentations  that  may  result  from  poorly  combined,  or  inappropriate  uses  of,  individual  screen  design 
elements.  Identifying  and  labeling  presentation  inlerfercnec  is  one  step  toward  the  development  of  both  belter 
screens  and  a contextual  approach  to  multiple  element  screen  design  research.  The  purpose  of  this  study  was  to 
determine  the  negative  effects  of  presentation  interference  and  to  lay  the  groundwork  for  future  research.  In 
addition,  the  results  of  this  study  will  contribute  to  research -based  guidelines  and  a presentation  evaluation  method 
that  can  be  used  during  and  after  presentation  development  to  help  designers  identify  and  eliminate  interference. 

Justification 

Existing  research-bused  prescriptions  lend  not  to  go  beyond  studies  of  single  elements  (e.g.,  text  size). 

Screen  design  researchers  have  avoided  examining  the  effects  of  more  than  a single  element  for  two  primary 
reasons:  (a)  when  individual  elements  arc  combined,  effects  cannot  be  clearly  attributed  to  one  factor  or  another;  and 
(b)  because  effective  screen  design  is  not  context  neutral,  screen  design  from  an  effective  presentation  cannot 
automatically  be  imposed  on  another  topic.  However,  these  typical  generalization  methods,  albeit  ostensibly  tidy, 
arc  not  very  realistic.  Actual  presentations  contain  more  one  element  and  generally  more  than  one  type  of 
presentation  design  flaw  (referred  to  here  as  presentation  interference).  Further,  the  combination  of  elements  may 
produce  an  effect  greater  than  the  sum  of  effects  from  elements  examined  individually  (Mukherjee  & Edmonds. 

1994).  Existing  single-clement  studies  have  tried  to  attribute  learning  gains  to  a specific  manipulation  of  an  element 
being  studied  (e.g.,  1 8 point  bold  text  versus  14  point  plain  text),  often  with  recommendations  that  the  element  be 
handled  by  other  developers  in  exactly  the  same  way  as  produced  the  better  result  in  the  particular  study.  Some 
studies  are  misleading  because  even  the  treatments  with  the  best  results  were  not  designed  according  to  professional 
layout  standards.  Further,  most  guidelines  encourage  novice  designers  to  follow  positive  statements  (e.g..  “use 
complimentary  fonts”).  Such  statements  create  problems  because  designers  may  think  they  have  followed  the 
directions,  yet  still  produce  poor  designs.  With  typical  prescriptive  guidelines,  designers  have  no  external  source  of 
feedback  or  evaluation. 

The  present  study  has  taken  the  reverse  approach  to  the  problem.  Rainer  than  comparing  two  “realistic 
practice”  treatments  and  prescribing  the  one  that  produces  the  higher  gain,  the  present  study  measured  the  results 
from  an  interfere  nee -free  presentation  design  against  results  from  versions  representing  “realistic  practice.”  The 
difference  between  what  was  learned  from  flawed  versions  and  what  could  have  been  learned  from  the  error  free 
version  is  referred  to  in  this  study  as  “learning  cost.”  By  identifying  and  measuring  the  lost  opportunity  for  learning 
that  results  from  screen  design  flaws,  evaluation  guidelines  can  be  produced  that  help  presentation  developers 
identify  and  eliminate  errors  in  presentations  that  may  distract  learners  from  the  messages.  Such  guides  (e.g., 

Bradshaw  1997b) would  fit  naturally  o formative  and  summativc  evaluation  processes. 

Relevant  Issues 

Visual  Literacy 

Any  research  on  screen  design  must  be  informed  by  the  field  of  visual  literacy.  Conversely,  findings  from 
screen  design  should  add  to  and  support  this  relatively  young  field.  After  only  25  years  of  research  and  exploration, 
the  field  of  visual  literacy  and  its  constructs  and  vocabulary  are  not  yet  fully  explored.  Dwyer  (1972)  developed  a 
program  of  research  to  identify,  classify,  and  measure  “essential  stimuli  characteristics”  of  media.  Braden  and 
Hortin  (1982,  cited  in  Hortin,  1994)  attempted  to  categorize  and  define  the  subsets  of  visual  literacy.  Baca  (1990, 
cited  in  Braden,  1994)  surveyed  52  visual  literacy  experts  in  an  attempt  to  identify  constructs  of  visual  literacy. 
From  this  study,  Baca  provided  one  of  the  simplest  and  most  recent  definitions  of  visual  literacy  to  refer  to  the  use 
of  visuals  for  six  purposes:  communication,  thinking,  learning,  constructing  meaning,  creative  expression,  and 
aesthetic  enjoyment  (p.  3).  The  first  four  of  these  purposes  arc  immediately  relevant  to  screen  design  and  to  the 
educational  purposes  for  which  screen  design  is  employed.  Baca's  definition  reflects  the  pervasive  nature  of  visuals, 
both  in  and  beyond  education. 

Braden  and  Baca  (1991 ) presented  a conceptual  map  of  the  visual  constructs  identified  in  Baca’s  survey.  In 
a graphic  representation,  the  six  elements  of  Baca's  definition  radiate  from  "purpose."  Each  of  the  six  elements  can 
then  be  further  elaborated  in  separate  diagrams.  For  example,  "visual  literacy"  used  to  describe  visuals  for  the 
purpose  of  learning,  can  be  expanded  in  a separate  diagram.  The  secondary  diagram  can  illustrate  the  goals  of 
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learning  including,  among  other  goals,  developing  the  ability  to  create  visuals  and  the  ability  to  acquire  information 
from  visuals  (p.  158).  Both  of  these  particular  goals  have  strong  links  to  another  of  the  six  basic  purposes  of  Baca's 
definition,  aesthetic  enjoyment. 

Aesthetics  serve  a greater  purpose  than  simple  enjoyment.  Aesthetic  principles  applied  to  screen  design 
may  make  learning  more  accessible  by  structuring  and  organi/.ing  information.  Visual  literacy  not  only  allows  both 
students  and  teachers  the  ability  to  understand  visual  information,  but  also  the  ability  to  develop  and  present 
information  that  is  more  accessible  to  other  viewers.  Fredette  (1993)  stated,  “Aesthetic  ways  of  knowing  must 
become  part  of  the  basic  education  of  all  students”  (p.  66).  Visual  literacy  is  vital  in  a time  when  most  teachers  have 
access  to,  and  encouragement  to  use,  programs  such  as  PowerPoint  to  aid  in  the  presentation  of  information.  Art  and 
aesthetics  may  he  as  important  to  educating  and  learning  as  arc  reading,  writing  and  arithmetic.  Several  researchers 
(e.g.,  Bcrtolinc,  Burton,  & Wiley,  1992)  have  recommended  that  visual  literacy  be  incorporated  into  curricula  or 
specific  courses,  including  mainstream  communication  courses.  For  at  least  two  decades,  educators  such  as  Brmidy 
(1977),  who  asked,  “Is  ’Pt  [art]  the  4th  R?”  have  called  for  a more  central  role  of  visual  literacy  in  education. 

Although  neither  visual  literacy  broadly  nor  screen  design  specifically  have  an  overt,  widespread  presence 
in  education,  screen  design  and  presentation  issues  may  well  he  covertly  operating  and  affecting  learners’  abilities  to 
understand  and  learn  from  material  presented  in  classrooms.  In  all  subject  areas,  educators  arc  feeling  pressure  from 
administration  and  peers  to  include  technology  in  their  classrooms.  Electronic  presentations  offer  one  of  the  most 
visible  and  accessible  ways  to  meet  this  demand.  Instructor-controlled,  computer-based  presentations  are  an 
intermediary  step  between  the  traditional  lecture  and  more  broad  incorporation  of  technology,  such  as  using 
computers  as  the  main  delivery  method  for  instruction,  tutoring,  and  practice.  If  the  visual  arrangement  of 
information  on  computer  screens  affects  learning,  aesthetics  must  he  acknowledged  and  incorporated  into  screen 
design. 

Identifying  Appropriate  Vocabulary 

Can  poorly-designed  presentation  screens  produce  negative  learning  effects?  If  so,  what  are  those  ell'ects  on 
learners'  comprehension  and  retention  of  message  content?  In  asking  the  question,  it  becomes  clear  that  one  of  the 
factors  that  may  he  hindering  the  movement  into  research  on  the  overall  effects  of  the  combined  elements  of  screen 
design  is  vocabulary.  How  docs  one  refer  to  the  negative  phenomena  resulting  from  poorly  comhined  individual 
elements  of  screen  design?  The  terms  studyability,  comprehensibility,  noise,  and  interference  arc  discussed  below. 

The  term  studyability  was  introduced  by  Grabingcr  (1993)  to  describe  “the  ease  with  which  a user  can 
examine  and  learn  from  a screen  of  information”  (p.  37).  The  term  has  not  been  widely  adopted  by  other  researchers 
and  is  somewhat  limited  in  that  a concentration  of  effort  on  the  part  of  the  learner  is  implied  that  is  not  always 
present. 

Comprehensibility 

The  term  comprehensibility  has  been  used  to  describe  the  overall  effectiveness  of  a screen  to  transmit 
information  to  viewers  (Grabingcr,  1989;  Pettersson,  1994).  The  term  measures  the  positive  cl  feels  of  the 
arrangement  of  screen  elements.  A high  degree  of  comprehensibility  would  mean  that  the  message  of  a particular 
screen  is  easy  to  access. 

Noise 

Tlic  term  noise  is  sometimes  used  to  describe  visual  phenomena  that  interfere  with  delivery  of  a message. 
In  their  review  of  the  literature,  Mukherjee  and  Edmonds  noted  that  “one  does  not  have  to  look  far  to  find  computer 
interfaces  and  screens  that  generate  ‘noise’  and  hinder  instruction  and  the  communication  of  messages”  (1994,  p. 
112).  Shannon  and  Weaver  used  this  term  in  their  model  of  communication  to  describe  phenomena  that  alter 
messages  so  that  what  is  received  is  different  from  what  was  sent.  For  lack  of  a widely  accepted  term,  tu»L\e  is  useful 
hut  can  become  less  so  when  applied  to  audiovisual  or  multimedia  applications  because  discussion  of  metaphorical 
noise  can  he  confused  with  literal  audio  noise. 
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The  word  interference  is  significant  in  several  fields.  In  cognitive  science,  the  term  refers  to  prior 
knowledge  that  interferes  with  the  acquisition  of  new  knowledge.  Cognitive  science  studies  show  that  interference 
turns  attention  away  from  the  message.  The  term  is  also  used  in  communication.  In  broadcasting,  for  example,  it  is 
used  to  refer  to  the  noise  resulting  from  poor  signal  transmission  that  interferes  with  clear  reception.  Whereas  noise 
implies  a change  in  the  message,  interference  suggests  the  possibility  that  the  message  may  not  even  be  received. 
Interference  is  not  without  precedent  in  visual  communication.  Bcrtolinc,  Burton,  and  Wiley  (1992)  stale, 

Interference  is  the  undcsircd  outside  influence  that  tends  to  prevent  the  completion  of  effective 

communication.  It  can  occur  at  any  level  or  stage  in  the  process.  The  goal  of  any  communication  is  to 

eliminate  or  minimize  the  influence  of  interference  and  thus  maximize  effective  communication,  (p.  252) 

Pcttcrsson  (1993)  uses  the  term  in  discussing  the  need  to  avoid  unnecessary  elements  in  visual  design:  “Too  many 
details  and  too  much  complexity  give  rise  to  distracting  ‘interference'  and  reduce  the  interest  for  the  content  and  the 
impact  of  the  important  part  of  the  content  of  the  visual”  (p.  227). 

Interference  is  a particularly  useful  term  that  is  not  limited  exclusively  to  one  medium.  The  term  works 
equally  well  in  describing  print-based  or  computer-mediated  instruction,  two  areas  often  seen  as  unable  to  apply 
identical  solutions.  Interference  can  be  applied  easily  to  projected  presentations  or  multimedia  applications  to 
describe  any  element  of  the  program,  visual  or  audio,  that  hinders  message  reception.  Where  comprehensibility  can 
be  used  to  deserve  the  overall  positive  effect  of  visual  arrangement,  interference  describes  the  individual  or 
combined  elements  that  negatively  affect  visual  or  audiovisual  designs. 

Several  types  of  flaws  (i.c.,  inappropriate  color  choice,  unreadable  text,  design  Haws  in  charts  and  graphs, 
etc.),  or  combinations  of  Haws,  contribute  to  poor  overall  design.  These  types  of  design  errors,  referred  to  here  as 
presentation  interference , have  a potentially  negative  effect  on  learning  from  visually  supported  presentations,  for 
two  primary  reasons.  First,  screens  that  arc  well  designed  provide  for  easier  cognitive  organization  (e.g.,  Hannafin  & 
Peek,  1988);  the  learning  process  itself  is  easier.  Second,  gaining  and  maintaining  learner  attention  is  a vital 
component  of  instruction  (e.g.,  Fleming  & Levic.  1978).  When  learners  form  negative  attitudes  toward  presentations 
or  attention  is  misdirected  toward  irrelevant  components,  learners  may  become  less  engaged  in  the  learning. 

Presentation  interference  includes  any  delivery-  or  presentation-related  factor  that  distracts  the  learner  from 
the  message  content  (Bradshaw,  1996a).  Inappropriate  uses  of  specific  elements  can  lead  to  presentation 
interference,  some  examples  of  which  include 

• meaningless  or  inconsistent  sound  effects  or  screen  transitions 

• inappropriate  colors  (e.g.,  color  coding,  fit  to  subject,  fit  with  other  colors) 

• inappropriate  typography  choices  (e.g.,  size,  font,  style) 

• inappropriate  graphic  choices  (e.g.,  size  style,  format) 

• inappropriate  placement  of  elements 

• inappropriate  or  inconsistent  alignment  of  elements 

The  term  "presentation  interference”  is  also  applicable  to  portability-related  distractions.  Elements  such  as  color  or 
graphical  characters  that  arc  appropriate  for  one  cultural  audience,  for  example,  may  he  extremely  distracting  for 
another  (Bradshaw,  1996b).  Necessarily,  presentation  interference  is  context  driven.  A more  thorough  guide  to 
suggested  categories  of  presentation  interference,  including  examples  and  a rubric  for  use  in  screen  design  critiques 
has  been  developed  and  used  by  the  author  (Bradshaw,  1997b). 

Poorly  Designed  Visuals 

Some  educational  researchers  view  the  lack  of  visual  or  instructional  training  as  just  two  more  problems  in 
a long  list  of  unprofessional  uses  of  instructional  tools  that  will  have  little  effect  on  the  legitimate  work  of  those 
trained  in  educational  technology  (e.g.,  G.  Morrison,  personal  communication.  February  13,  1995).  Others, 
particularly  those  with  interests  in  visual  literacy,  arc  concerned  that  the  "slick  feel"  of  modern  media  lulls  users 
(both  teachers  and  developers)  into  accepting  many  presentations  and  software  programs  as  instructional ly  "valid" 
and  "good,"  regardless  of  their  actual  educational  worth.  When  computer  presentations  arc  delivered  via  projection 
systems,  such  as  LCD  (liquid  crystal  display)  projectors  and  panels,  the  problem  is  exacerbated.  This  investigator’s 
experience  has  been  that  the  use  of  this  equipment  may  exaggerate  the  effectiveness  the  presentation  in  the  mind  of 
the  presenter  without  necessarily  improving  learning. 


Much  remains  to  be  explored  regarding  the  learning  costs  of  poorly  designed  visuals  in  instruction.  II* 
visual  design  can  affect  learning,  instructional  designers  must  consider  both  the  delivery  method  and  appearance  of 
instruction  in  addition  to  the  content  and  strategy.  Haag  and  Snetsigner  (1994)  agree  that,  “as  form  and  function  go 
hand  in  hand  so,  too,  must  instructional  design  and  aesthetic  design.  They  are  inextricably  connected”  (p.  94). 

Hardin  (1994)  has  suggested  the  problem  of  poorly  designed  visuals  could  be  solved  if  teachers  would 
simply  practice  developing  materials.  Regarding  the  production  of  charts  and  diagrams,  Hardin  said,  “we  should 
encourage  non-designers  to  produce  frequently  on  the  theory  that  practice  will  increase  skill1’  (p.  24).  But  practice 
docs  little  good  without  credible,  empirically  based  guidelines.  Practicing  visual  design  with  neither  attention  to 
visual  design  principles  nor  expert  feedback  is  just  as  likely  to  reinforce  bad  habits  and  poor  techniques  as  it  is  to 
develop  expertise.  The  argument  is  somewhat  moot,  since  few  teachers  have  the  luxury  of  spending  time  practicing 
designs  that  they  have  no  intention  of  actually  using. 

Screen  Design 

In  their  review  of  the  literature  on  screen  design,  Mukherjee  and  Edmonds*  (1994,  p.  115)  noted  that  the 
general  functions  of  screen  design  arc  to 

• focus  learners’  attention 

• develop  and  maintain  learners’  interest 

• promote  deep  processing 

• promote  engagement  between  learners  and  lesson  content 

• help  learners  find  and  organize  information 

Instructional  design  journal  articles,  chapters,  and  hooks  focus  on  the  above  screen  design  functions  and 
occasionally  focus  on  the  role  of  specific  screen  design  elements  in  achieving  these  goals.  Hannafin  and  Peek 
(1988),  for  example,  link  screen  design  to  purpose  and  screen  type:  transitional,  instructional,  and  question.  In  many 
instructional  design  books  (c.g.,  Criswell,  1989),  specific*  ourposc  screens  (e.g.,  title,  menu,  instruction,  practice, 
feedback)  arc  emphasized  more  than  the  particular  visual  design  elements  within  these  screens.  Emphasis  is  given  in 
general  terms,  such  as  to  the  need  for  standard  screen  components  (e.g.,  navigation  controls),  functional  areas,  (i.e., 
where  titles  should  appear  and  what  portion  of  the  screen  should  be  reserved  for  graphics),  and  types  of  visuals  (e.g., 
charts,  graphs,  etc.).  Guides  of  the  type  described  arc  not  generally  accessed  by  most  “nonprofcssional”  screen 
developers  and  arc  intended  for  “front-end”  use.  However,  presentation  media  developers,  many  of  whom  arc 
teachers  who  develop  their  own  materials  by  necessity,  also  need  a system  that  can  be  used  in  a follow-up  evaluation 
process,  one  that  is  specific  enough  to  catch  design  flaws  while  being  flexible  enough  to  accommodate  specific 
contexts. 

Lack  of  Research-based  Guidelines 

Although  much  screen  design  literature  provides  strategics  for  screen  design,  Mukherjee  and  Edmonds 
(1994)  found  that  most  design  strategics  are  not  supported  by  research  evidence  or  experience  (p.  1 13).  Some  design 
texts  arc  supported  in  small  part  by  research,  yet  even  in  these  eases  there  arc  often  unattributed  statements  such  as 
"black  and  yellow  are  the  most  readable  combination.”  While  statements  such  as  these  may  well  be  backed  by 
research,  citations  arc  not  provided.  There  is  a conspicuous*  lack  of  research  to  support  the  intuitive  sense  that 
instruction  that  looks  good  should  make  learning  easier.  Even  within  the  research  community,  not  all  of  the  overall 
guidelines  (that  address  more  than  a single  element)  arc  entirely  based  on  research  and,  where  guidelines  arc 
research  based,  specific  guidelines  are  not  always  linked  to  specific  studies  (e.g.,  Fleming  & Lcvic,  1978).  Examples 
of  non-rcscarch-bascd  or  partially  research-based  prescriptive  guidelines  include  those  by  Keyes  (1993),  Len/.e 
(1992),  Pettersson  (1993),  and  MUheim  and  Lavix  (1992).  Kosslyn’s  (1989)  thorough  article  “Understanding  Charts 
and  Graphs,”  which  points  out  design  flaws  and  provides  cognitive  rationales  for  design  choices,  is  an  excellent 
guide  for  the  development  of  charts  and  graphs  but  docs  not  provide  research  evidence  to  support  prescribed  design 
choices.  Misanchuk  (cited  in  Grabingcr,  1993)  states  that  while  aesthetic  guidelines  exist  to  help  designers  create 
attractive  displays  “aside  from  Hartley’s  (1978)  work,  there  arc  few,  if  any,  empirically  based  guidelines  to  help 
instructional  designers  combine  text  elements  in  ways  that  facilitate  learning”  (p.  35). 

Practical  Guidelines 

Achieving  screen  design  goals  using  specific  visual  design  elements  has  been  the  focus  of  several 
textbooks,  such  as  Rabb's  (1990)  Presentation  Design  Book,  although  most  visual  design  texts  have  hecn  written 
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primarily  for  prim  media  (Hariley,  1978),  Typical  examples  of  guides  lo  visual  design  include  desktop  publishing 
and  basic  design  books,  such  as  Text,  Typography  and  Layout  for  Desktop  Publishing  (Silver  & Silver,  1991). 
Desktop  publishing  software  producers  often  develop  their  own  layout  and  design  guides.  Aldus  published  a basic 
design  book  (Kcrlow,  1991)  that  included  tutorials  for  PageMaker  4.0,  A wide  variety  of  magazine  and  journal 
articles  have  also  been  written  to  aid  the  growing  population  of  novice  software  producers,  such  as  Olson  and 
Wilson’s  (1985)  simple  checklist  for  designing  computer  screen  displays  and,  more  recently,  guides  such  as  those  by 
McFarland  (1995),  Wilhelm  (1996),  and  Strasscr  (1996),  and  guidelines  available  via  the  internet  (c.g.,  Radcl,  1995) 

As  electronic  media  become  more  accessible,  more  prescriptive  articles  appear  in  a wider  variety  of 
journals  and  magazines.  The  majority  of  these  works  rely  on  common  practice  and  “common  sense,"  as  opposed  to 
findings  from  research.  One  of  the  problems  with  approaching  the  concept  of  "good  screen  design"  is  that  there  is  no 
possibility  of  an  ultimate  solution.  "Good  design"  is  not  context  neutral  - specific  arrangements  that  work  well  for 
one  content  area  and  audience  do  not  automatically  work  well  for  all  other  content  areas  and  audiences.  Still,  this 
has  not  stopped  the  production  of  prescriptive  guidelines  calling  for  specific  text  sizes  and  color  combinations  that 
should  be  used  in  any  case. 

General  Screen  Design  Prescriptives 

Categorizing  specific  screen  design  elements  and  then  manipulating  elements  within  each  category  has 
been  referred  to  by  Mayer  (1984)  as  a “behavioral”  approach.  This  has  been  one  of  the  most  common  ways  to 
approach  an  initial  understanding  of  screen  design  and  is  the  method  used  in  most  existing  screen  design  research. 
Although  a useful  place  lo  start,  the  method  is  limited  because  general  prescriptives  arc  not  equally  relevant  lo  all 
settings.  Much  of  the  existing  prescriptives  focus  on  the  following  elements:  text,  graphics,  color,  line,  animation, 
and  consistency.  The  prescriptives  that  follow,  allhough  not  exhaustive,  arc  typical  of  those  found  in  the  literature. 
They  arc  based  on  professional  practice,  general  heuristic,  and  research. 

Text 

Several  researchers  (e.g.,  Hathaway  1984;  Mourant,  Lakshmanan  , & Chaniadisai,  1981)  report  that  large 
amounts  of  text  cause  eye  fatigue.  Researchers  (Hartley,  1987;  Hathaway,  1984;  Morrison,  Ross,  & Dell,  1988,  cited 
in  Knupfer,  1994)  also  report  that  large  amounts  of  text  arc  more  likely  to  be  forgotten.  Soulier  (1988)  suggests  lha 
when  a great  deal  of  text  is  called  for,  it  is  better  to  print  the  text  out. 

Text  displayed  in  all  capital  letters  is  difficult  to  read  (Pettersson,  1993),  therefore,  in  most  eases,  text 
should  contain  a mixture  of  upper  and  lower  ease  characters.  Ross  and  Morrison  (1988)  also  suggest  using  low- 
density  text,  reducing  ideas  to  one  idea,  and  using  outline  format.  In  many  eases,  minimal  drop  shadows  can  make 
text  more  legible. 

Regarding  the  use  of  serif  or  san  serif  fonts,  suggestions  from  the  literature  arc  mixed.  It  is  generally 
accepted  that  what  readers  arc  used  to  reading  or  personally  prefer  will  be  easier  for  them  to  read  (Dreyfus,  1985). 
Some  researchers  suggest  that  small  serifs  should  be  used  for  printed  material  and  sans  serifs  be  used  for  computer 
and  projected  material  (because  thin  serifs  do  not  generally  project  well). 

Gibson  and  Mayla  (1992,  cited  in  Knupfer,  1994)  suggest  that  for  computer-displayed  text,  text  size  should 
not  be  smaller  than  26  points.  This  prescriptive  creates  a problem,  however,  because  not  ail  26-point  fonts  arc 
visually  the  same  size.  For  computer-displayed  text,  Gibson  and  Mayta  also  suggest  using  bold  text  throughout,  or  at 
least  for  headings  and  key  words.  Again,  this  must  depend  on  the  specific  typeface  used  because  some  fonts  arc 
already  "heavy"  and  bolding  may  actually  make  them  less  readable. 

Graphics 

Soulier  (1988)  describes  graphics  as  visual  metaphors.  Graphics  aid  in  motivation  and  maintaining 
attention  by  adding  variety,  thereby  making  the  screen  more  interesting  (Kemp  & Dayton,  1985).  Aspillaga's  ( 1991 ) 
research  suggests  that  placing  graphics  close  to  corresponding  text  results  in  significantly  higher  achievement  than 
placing  them  elsewhere  on  the  same  screen.  In  cases  where  information  must  be  split  to  two  screens,  Soulier  (1988) 
suggests  placing  the  graphics  after  the  text.  This  contradicts  Pellersson’s  (1993)  suggestion  that  developers  achieve 
priming  by  placing  graphics  before  related  text. 

Researchers  (c.g..  Pettersson,  1993;  Fleming  & Levic,  1978;  and  Soulier,  1988)  report  that  simple  line 
drawings  arc  generally  more  effective  than  complex  drawings  or  photographs  because  they  contain  only  relevant 
information;  less  information  requires  less  time  to  interpret. 
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Color 


In  general,  the  most  highly  preferred  surface  colors  for  adults  arc  blue,  red,  and  green  (Pettersson,  1993). 
Generally  ranked  lowest  are  violet,  orange,  and  yellow  (Fleming  & Levie,  1978).  The  most  visible  colors  are  white, 
yellow,  and  green  (Pettersson,  1993).  The  least  visible,  assuming  equal  intensity,  arc  red  and  blue  (Fleming  & Levie, 
1978).  Researchers  (c.g.,  Pettersson,  1993;  Fleming  & Levie,  1978)  agree  that  the  most  readable  color  combination 
is  black  and  yellow.  A good  general  guideline  is  to  start  with  a fairly  light  or  fairly  dark  background  color  that  is 
appropriate  to  the  content,  and  then  choose  an  opposite  (fairly  dark  or  fairly  light)  color  for  text  (Bradshaw,  1997a). 
This  general  strategy  allows  for  color  combinations  that  arc  appropriate  to  the  specific  topic  and  allows  for  more 
creativity  while  still  ensu.  *g  readability.  Rather  than  focusing  on  specific  colors  as  always  best,  developers  should 
work  to  ensure  contrast  between  background  and  text. 

Dwyer  (e.g.,  1972,  1978)  has  conducted  many  studies  on  color  and  reports  that  color  coding  and  cueing 
increase  learning.  Hannafin  and  Peck  (1988)  suggest  bright  colors,  such  as  red,  for  cueing  viewers  to  new 
information;  however,  developers  should  focus  more  on  "bright  color"  in  this  statement  than  on  "such  as  red," 
because  red  is  one  of  the  most  difficult  colors  to  see  (Fleming  & Levie,  1978;  Pettersson,  1993).  "Bright  color" 
could  also  include  yellow,  one  of  the  most  visible  colors.  Again,  context  must  be  considered.  A light,  bright  color 
used  for  highlighting  would  not  be  successful  on  a light  background  for  instance. 

Line 

Lines  carry  information  by  way  of  location  of  point  of  origin,  curvature,  direction,  length,  point  of  change, 
or  terminus  (Fleming  & Levie,  1978).  Straight  lines  arc  very  quickly  perceived,  compared  to  less  regular  and 
predictable  lines  (Fleming  & Levie,  1978).  Horizontal  and  vertical  lines  are  perceptually  "special."  They  are  more 
intense  (i.e.,  they  evoke  more  activity  in  the  visual  cortex  of  the  brain),  arc  more  easily  compared,  and  arc  more 
accurately  judged  for  spatial  orientation  (Fleming  & Levie,  1978). 

Animation 

Soulier  (1988)  recommends  using  animation  only  when  appropriate  and  keeping  it  short.  In  a study  of  the 
effects  of  visual  grouping  strategies  with  animated  and  static  graphic  presentations,  Rc*ber  (1991)  found  that 
“animation  was  a significant  aid  to  learning  as  compared  to  static  graphics,  but  only  when  presented  part  of  a 
scries  of  verbal  and  visual  chunks”  (p.  12).  The  literature  docs  not  provide  evidence  that  animation  for  its  own  sake 
increases  learning.  By  extension  of  the  simplicity  principle  that  applies  to  graphics,  simple  animations  arc  often 
more  effective  than  video,  although  they  do  not  have  the  same  emotive  potential  as  video. 

Consistency 

Thurling,  Hanneman,  and  Haakc  (1995)  discuss  issues  relating  to  consistency  and  use  the  term  "cognitive 
overhead"  to  describe  the  process  and  effort  required  to  make  sense  of  each  new  screen  when  elements  arc  not 
arranged  and  placed  consistently  throughout  a program.  They  suggest  cognitive  overhead  can  he  reduced  and 
attention  to  the  message  increased  through  careful  attention  to  consistency  within  all  categories  of  screen  design. 
Consistency  of  navigation  and  help  procedures,  as  well  as  consistent  location  of  certain  classes  of  elements  (c.g., 
diagrams  or  text  blocks)  is  recognized  to  simplify  the  learning  task  (Hannafin  & Peek,  1988). 

Pertinent  Studies 

Research  on  the  visual  elements  of  screen  design  is  generally  divided  into  single-element  or  multiple- 
element  studies. 

Single-element  Studies 

The  vast  majority  of  research  in  the  area  of  screen  design  has  focused  on  the  effects  of  individual  design 
factors  (c.g.,  text  size).  Many  of  the  single-element  studies  include  examples  of  presentation  interference,  although 
the  researchers  viewed  their  studies  in  terms  of  positive  results  from  specific  prescribed  practices  rather  than  in 
terms  of  negative  effects  resulting  from  intcrfcrenec.  Dwyer  (c.g.,  1978)  has  extensively  studied  visualization  and 
the  effects  of  color,  color  coding,  and  realism.  Several  of  his  studies  have  centered  on  the  relative  effectiveness  of 
different  types  of  illustrations  in  facilitating  learning.  For  example,  in  one  study  subject  groups  were  shown  different 
presentations  that  included  the  same  instructional  content  hut  with  different  visual  representations  of  a heart.  Visuals 
ranged  from  simple  black  and  while  line  drawings  to  realistic  color  photographs.  Dwyer  provides  samples  of  the 
heart  visuals  used  (1978,  p.  56).  These  visuals  provide  an  example  of  how  interference  may  he  unintentionally 
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included  in  visual  research.  One  of  the  visuals  used  was  a color  photograph  of  a heart,  which  was  very  clear  and 
realistic.  Another  visual,  a lower-quality  black  and  white  photograph  of  the  same  heart,  is  not  realistic  or  even 
recognizable,  yet  the  two  were  used  to  compare  the  differences  in  learning  from  color  or  black  and  white  visuals. 
This  example  is  less  a comparison  between  color  and  black  and  white  than  it  is  an  example  of  a poor  visual 
interfering  with  delivery  of  a visual  message.  Despite  the  visual  bias  of  this  particular  study  and  others  like  it, 
research  on  the  actual  benefits  of  color  to  learning  remains  inconclusive  (Dwyer,  1978,  pp.  147-150). 

Knupfer  and  Mclsaac  (1990,  1991)  tested  retention  of  subject  matter  when  text  was  interrupted  with  a 
graphic  and  varying  degrees  of  white  space.  A central  question  of  the  initial  study  was  “at  what  point  does  the 
amount  of  white  space  surrounding  a graphic  enhance  or  inhibit  reading  of  the  text?”  The  authors  point  out  that 
because  materials  used  in  the  studies  were  not  professionally  produced,  the  arrangement  of  text  and  graphics  could 
be  assumed  to  reflect  common  practice.  This  is  quite  credible  and,  in  fact,  the  layout  used  was  not  unlike  many 
arrangements  found  by  this  investigator.  Specifically,  a graphic  was  inserted  within  a single  column  of  text,  which 
made  reading  more  challenging  because  the  reader  had  to  search  for  the  next  word  following  a break  in  text.  As  the 
studies  showed,  the  more  white  space  surrounding  the  graphic,  the  lower  the  gain  between  pretest  and  posttest  on  the 
material  read.  White  space  used  in  this  way  interferes  with  the  message  content  (one  way  to  remedy  this  type  of 
interference  is  to  break  the  single  wrap-around  column  into  two,  with  the  text  from  each  column  wrapping  around 
only  one  side  of  the  graphic).  These  studies  arc  quite  useful  because  they  illustrate  how  white  space,  generally 
thought  of  as  desirable,  can  easily  contribute  to  interference. 

Knupfer  and  Mclsaac  (1992)  conducted  a similar  study  to  examine  the  effects  of  graphics  with  varying 
degrees  of  shading  superimposed  on  a text  field.  Subjects,  divided  into  four  groups,  read  displayed  text  and  were 
then  tested  on  the  material  they  read.  Subjects  reading  text  with  no  graphic  scored  higher  than  those  reading  text 
with  an  overlaid  line  drawing.  A third  group,  who  read  text  with  an  overlaid  graphic  with  10 9c  gray  shading,  scored 
even  lower.  Although  the  performance  of  these  first  three  groups  suggests  support  for  the  belief  that  interference 
negatively  affects  learning,  a fourth  group,  who  read  text  with  an  overlaid  graphic  with  40%  shading,  scored  higher 
than  the  group  presented  with  only  10%  shading.  It  may  be  that  where  certain  types  of  interference  are  very  high  and 
where  certain  motivations  (c.g..  immediate  testing  or  grades)  exist,  subjects  exert  greater  effort  to  perceive  the 
message. 

Studies  of  individual  screen  elements  are  important  and  necessary  because  they  provide  a starting  point  for 
research  into  screen  design  as  a whole.  Understanding  the  effects  of  individual  elements  enables  a designer  to 
combine  them  in  ways  that  arc  theoretically  effective.  However,  the  benefit  of  these  studies  is  limited  because  in 
actual  practice,  usually  more  than  a single  element  of  the  overall  design  is  flawed.  Examining  only  one  factor  at  a 
time  docs  not  represent  what  Grabingcr  (1989)  refers  to  as  the  visual  "gestalt."  The  problem  is  staled  well  by 
Mukherjec  and  Edmonds  (1994):  “even  when  it  is  known  which  elements  have  been  combined,  the  overall  effect  of 
the  combination  is  not  equal  to  the  sum  of  the  effects  of  each  individual  element"  (p.  1 15).  Neuman  (1991 ) further 
comments  on  a fundamental  limitation  of  single-element  research. 

Realit  . . is  indivisible  as  well  as  multiple.  Residing  wholly  in  an  individual’s  mind,  reality  cannot  be 
fragmented  into  variables  to  be  studied  in  isolation.  Separating  any  part  from  the  whole  invariably  alters 
both  the  part  and  the  whole:  studying  only  discrete  parts  therefore  distorts  the  reality  we  seek  to  understand. 

( p.  41) 

Necessarily,  screen  design  works  synergisticallv  for  each  viewer. 

Multiple-element  Studies 

Multiple-element  screen  design  studies  are  almost  nonexistent.  This  type  of  research  is  difficult  because 
uhen  individual  elements  arc  combined,  effects  cannot  he  clearly  attributed  to  one  factor  or  another.  Grabingcr 
(1989)  found  that 

Despite  research  into  individual  screen  design  factors,  there  is  a dearth  of  research  into  effects  of 
combinations  of  these  factors.  Multiple-clement  research  lends  to  be  more  complex  than  single-element 
research.  Examination  of  single  text  elements  usually  stops  with  that  element  and  its  effects  on  narrowly 
defined  tasks,  (p.  178) 
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Grabingcr  (1993)  attempted  to  move  beyond  single-element  studies  by  presenting  many  arrangements  and  asking 
viewers  to  select  the  ones  they  found  most  readable.  The  visuals  used  in  the  study  consisted  of  rectangles  to 
represent  the  location  of  graphics  and  “X”s  to  represent  text.  This  study  was  neither  gcncralizablc  nor  realistic 
because  the  visuals  had  no  context.  “Effective”  screen  design  is  not  context  neutral  - a screen  design  from  an 
“effective”  presentation  cannot  automatically  be  imposed  on  another  topic.  Results  from  studies  by  Morrison,  et  al 
(1989,  cited  in  Morrison  cl  al,  1991,  p.  189)  suggest  that  subjects  “may  apply  different  perspectives  when  evaluating 
screens  with  realistic  content  because  of  the  need  to  process  the  information  so  that  it  can  be  recalled  or  applied  at  a 
later  time.” 

An  earlier  study  on  the  effects  of  multiple  elements  of  screen  design  was  conducted  by  Grabingcr  and 
Albers  (1988,  cited  in  Grabingcr  1989)  in  which  fourth  graders  were  presented  with  CA1  programs  with  two 
different  screen  designs.  One  used  plain  text  and  the  other  incorporated  indentation,  highlighting,  command  bars, 
and  boxes  to  make  the  screens  appear  more  organized  and  structured.  The  researchers  found  no  difference  in  recall 
or  retention  between  the  two  groups,  but  they  did  note  a difference  in  the  average  lime  spent  on  each  screen. 

Time  is  a vitai  element  in  learning.  It  make  sense  that  learning  from  complex  or  unorganized  screens 
requires  greater  amounts  of  time.  In  user-controlled  CAI.  students  arc  able  to  accommodate  for  different 
presentation  styles  by  spending  more  time  reading  or  reviewing  screens  that  arc  difficult  to  comprehend.  This  extra 
time  is  not  available  in  content-centered  instruction,  such  as  in  tcaehcr-controllcd  or  prerecorded  presentations  to 
groups  of  learners.  In  these  eases,  students  do  not  have  the  extra  lime  to  make  up  for  lack  of  clarity,  nor  can  they 
return  to  specific  screens  for  review.  It  follows,  then,  that  where  the  purpose  of  a study  is  to  measure  the  negative 
effects  of  various  screen  designs  on  learning  and  retention,  lime  must’be  controlled.  This  condition  would  also 
reflect  much  of  the  actual  practice  of  classroom  presentations.  If  learners  do  not  have  control  over  time,  will  they 
learn  and  retain  even  less  of  the  instructional  content  from  poor  or  even  average  screen  designs  than  they  would 
from  interfere  nee- free  screens? 

Recommendations  From  the  Literature 

A clear  need  for  comprehensive  studies  exists.  Researchers  are  beginning  to  call  for  more  research  on  the 
effects  of  combinations  of  individual  elements.  Grabingcr  (1989),  suggests  the  need  for  researching 
recommendations  about  the  best  ways  to  combine  several  elements  (p.  178).  Haag  and  Snctsigncr  (1994)  believe 

Implementing  artistic  principles,  to  improve  the  aesthetic  appeal  of  screens  may  offer  a more  holistic  and 

comprehensive  method  of  testing  the  impact  of  screen  design  on  learners.  This  approach  may  prove  a better 

method  of  investigation  in  this  area  than  prior  studies  examining  individual  variables  (p.  96). 

Dwyer  ( 1994)  predicted  that,  at  the  college  level,  future  research  will  he  inullifocuscd  and  will  continue  to 
explore  the  instructional  effect  of  intervening  variables  (used  singly  and  in  combination).  Research  will  he  more 
comprehensive.  Dwyer  hopes  prescriptive  guidelines  will  he  produced  to  help  identify  generic  conditions  necessary 
to  maximize  learning.  He  also  suggests  that  different  types  of  visuals  and  different  types  of  cueing  need  to  he 
comprehensively  explored. 

Grabingcr  (1993),  Mukhcrjcc  and  Edmonds  (1994),  and  Haag  and  Snctsigncr  (1994)  believe  screen  design 
research  should  move  beyond  single  element  studies.  All  have  called  for  research  recommendations  regarding 
combining  several  elements  and  believe  the  time  has  come  for  multiple -element  studies.  Seels  (1994)  believes  that, 
for  research  in  visual  literacy  to  proceed,  the  constructs  of  visual  thinking,  learning,  and  communication  need  to  he 
associated  with  more  specific  operationally  defined  constructs. 

Realistically,  actual  presentations  will  contain  more  than  one  type  of  Haw.  Therefore,  finding  creative 
approaches  toward  accomplishing  multiple-element  research  is  worth  the  effort.  The  benefits  of  wcll-desir icd 
screens  and  the  costs  of  poorly  designed  screens  can  only  he  fully  understood  through  research  that  combines  design 
elements  in  realistic  ways.  Realistic  practice,  including  multiple  design  elements,  should  be  measured  against  a 
contextually  “ideal”  arrangement  of  elements  as  hypothesized  by  combining  findings  from  individual  clement 
studies  and  from  professional  practice.  Studies  on  individual  elements  have  been  and  continue  to  be  important. 
Without  them,  we  would  not  now  be  ready  to  embark  on  the  next  stage  of  research  on  screen  design.  With  the 
knowledge  gained  from  past  research,  wc  can  attempt  to  develop  interference-free  screens  and  hypothesize  about  the 
effects  of  combined,  flawed  elements.  A review  of  the  literature  suggests  that  research  into  the  area  of  screen  design 
should 

• acknowledge  the  "visual  gestalt" 
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• acknowledge  ihe  importance  of  context 

• measure  the  effects  of  combined  screen  elements  on  learning 

• have  a way  of  determining  whether  some  combinations  of  elements  arc  more  (or  less)  effective  in 
aiding  learning  than  others 

• be  based  on  treatments  of  visual  elements  previous  research  and  practice  have  suggested  arc  most 
effective 

• lead  to  research-based  prescriptive  guidelines 

• lead  to  operationally  defined  constructs  that  invite  further  research 

Method 

The  present  study  incorporates  the  above  recommendations.  In  addition,  this  study  (a)  includes  materials  designed 
according  to  professional  standards  that  have  been  critiqued  by  experts,  (b)  measures  subjects’  beliefs  regarding  the 
content  and  method  of  delivery,  and  (c)  contributes  to  a process  of  contcxt-bascd  screen  design  feedback  to  be  used 
during  and  after  presentation  development. 

Research  Hypotheses 

Three  hypotheses  were  investigated  during  this  study: 

Hypothesis  I:  A presentation  that  is  intentionally  presentation  interference  free  will  result  in  higher  postlcst  scores 
than  will  presentations  with  interference.  Hypothesis  2:  As  the  amount  of  presentation  interference  in  a presentation 
increases,  corresponding  posttest  scores  will  decrease.  Hypothesis  3:  Subjects  viewing  the  interference  free 
presentation  will  report  more  positive  beliefs  regarding  (a)  the  presentation,  (b)  computer  generated  presentations  in 
general,  and  (e)  instructors  who  use  computer  generated  presentations,  than  will  those  viewing  presentations 
containing  presentation  interference. 

Subjects 

Subjects  were  118  students  enrolled  in  undergraduate  Computers  in  Education  courses  at  a large 
Southwestern  university  during  the  fall  1996  semester.  Of  these,  74  (63%)  arc  female,  (86%)  were  within  the  age 
range  of  17-29;  9 subjects  were  aged  30  - 39;  5 subjects  were  aged  40  - 49;  and  2 subjects  were  over  50  years  old. 
Ninety  eight  of  the  subjects  (80%)  were  enrolled  in  one  of  three  sections  of  Computer  Literacy,  an  undergraduate 
course  with  some  emphasis  on  computers  in  education.  Twenty  five  subjects  were  enrolled  in  one  of  four  sections  of 
Computer  Applications,  a recommended  course  for  students  in  the  College  of  Education.  Although  not  required  of 
all  students,  both  classes  meet  a university  numeracy  requirement,  so  they  arc  viewed  by  many  students  as  required. 
Subjects  received  extra  credit  from  their  instructors  for  participating  in  the  study  and  were  randomly  assigned  to  one 
of  three  groups. 

Treatments  and  Instruments 

A computer  generated  presentation,  such  as  an  instructor  could  create  and  present  using  PowerPoint  to 
support  a speakers  presentation,  was  developed  via  Amhorware  to  teach  learners  the  various  types  of  skin  cancer 
and  how  to  prevent  them.  The  presentation  included  recorded  narration,  was  designed  according  to  known  research 
findings  and  prescriptive  guidelines,  and  was  12  minutes  in  length.  The  “control"  version  was  free  of  intentional 
interference  and  passed  a screen  design  critique  by  three  professional  presentation  designers  employed  at  a major 
electronic  technology  corporation  in  the  Southwest. 

In  the  second  version  of  the  presentation,  two  elements  were  intentionally  manipulated  in  order  to  produce 
presentation  interference.  Treatment  2 began  with  the  control  version,  with  two  changes:  (a)  background  color 
(changed  from  medium  blue  to  pink);  and  (b)  font  (changed  from  a basic  sail  serif  font  to  “Copacabana.”  a more 
ornate,  display  font). 

In  the  third  version,  four  changes  were  made.  Treatment  3 began  with  the  control  version,  to  which  the 
following  changes  were  made:  (a)  background  color  (same  as  version  2),  (b)  font  (same  as  version  2),  (c)  the 
addition  of  transition  sounds,  and  (d)  slightly  off-center  but  consistent  text  and  bullet  placement.  Although  the  point 
size  for  text  remained  constant  for  all  three  treatments,  the  x-height  of  the  font  used  in  Treatments  2 and  3 is  smaller 
than  the  x-height  of  the  font  used  in  Treatment  1.  The  result  is  text  that  is  technically  the  same  size  but  (hat  appears 
to  be  smaller.  Thus,  the  net  result  is  three  types  of  interference  present  in  Treatment  2 and  five  types  of  presentation 
interference  in  Treatment  3. 
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All  the  presentations  were  presented  via  Power  Macintoshes  with  17-inch  monitors  and  were  programmed 
to  control  the  amount  of  time  spent  viewing  each  screen.  The  liming  for  the  three  presentations  was  identical  except 
for  the  additional  time  required  in  version  three  by  the  length  of  the  sound  effects  used.  (The  sound  effects  ranged  in 
length  from  one  to  two  seconds.) 

A two-part  instrument  was  used  to  collect  data.  The  first  part  was  a 25-item  “short  answer”  and  “fill  in  the 
hlank”  achievement  test  regarding  the  information  presented.  The  second  portion  was  a series  of  12  seven-point, 
bipolar  probability  items.  Questions  1 through  5 of  the  belief  questionnaire  referred  to  students'  beliefs  regarding  the 
ercdibility  and  pacing  of  the  program  and  how  much  they  learned  from  it.  Questions  7 through  9 referred  to 
subjects'  beliefs  regarding  computer  generated  presentations  and  the  instructors  who  use  them.  Question  10  referred 
to  the  subjects’  own  experiences  with  screen  design  and  multimedia,  and  questions  6,  11,  and  12  referred  to 
subjects'  beliefs  about  their  behavior  before  and  after  viewing  the  presentation. 

Content  validity  was  assured  in  the  following  manner:  Content  for  both  tests  and  instruments  was  based  on 
information  and  photographs  distributed  by  the  American  Cancer  Society  and  the  Mayo  Clinic.  Posttest  items  arc 
parallel  to  the  presentation.  Both  test  and  treatments  were  developed  by  a university-level  presentation  design 
instructor.  Upon  completion,  the  presentation  was  evaluated  by  three  professional  presentation  developers  in  a blind 
critique.  Both  the  test  and  the  presentation  were  evaluated  for  content  by  the  researcher,  two  software  development 
teachers,  and  one  corporate  trainer. 

Most  internal  consistency  measures  require  multiple  administrations  of  a lest  or  multiple  forms  of  the  same 
test.  A split  half  internal  consistency  procedure,  which  can  be  calculated  after  a single  administration  of  a single 
form,  was  used  for  this  study.  Using  the  split  half  procedure,  the  reliability  coefficient  for  the  posttest  was  0.82. 

Procedures 

Three  versions  of  a prerecorded  instructional  presentation  were  delivered  via  Authonvare.  The  lesson 
centered  on  the  topic  of  skin  cancer  prevention.  The  presentation  was  prerecorded  in  order  to  control  the  amount  of 
time  spent  on  each  screen.  The  control  version  was  free  of  intentional  interference  and  the  treatment  versions 
contained  typical,  limited  combinations  of  interference.  Data  were  collected  during  a two-week  period.  Subjects 
chose  their  own  participation  limes  from  a variety  of  times  offered  and  were  randomly  assigned  to  one  of  three 
presentation  treatments.  Treatments  were  presented  via  Power  Macintoshes  with  17-inch  monitors.  Subjects  viewed 
the  presentations  individually,  wearing  headphones,  under  the  same  physical  conditions,  in  the  same  laboratory,  with 
the  same  lighting  conditions,  and  received  extra  credit  for  participating  in  the  study.  The  duration  of  the  presentation 
was  12  minutes.  All  subjects  completed  the  questionnaires  immediately  following  the  presentation  and  were  allowed 
as  much  time  to  complete  the  posttest  and  questionnaire  as  they  desired.  Completing  the  questionnaire  required 
between  10  and  20  minutes.  All  posttests  were  scored  by  the  same  grader  and  rcchcckcd  for  accuracy  by  another 
instructor.  The  number  of  points  correctly  answered  on  the  poshest  was  converted  to  a proportion  of  the  items 
possible. 

Design 

In  order  to  determine  the  effect  of  presentation  interference,  an  experimental  posttest  only  control  group 
design  was  used  (Figure  4).  The  posttest  only  control  group  design  with  random  group  assignment  controls  for  all 
sources  of  internal  invalidity  except  for  mortality  (Gay,  1992).  Mortality  was  not  a serious  threat  to  this  study 
because  of  the  short  data  collection  period.  Variables  such  as  age,  computer  anxiety,  grade  point  average,  etc.,  were 
controlled  for  by  random  assignment  to  treatment  groups.  An  interference  free  presentation  was  used  as  the  control 
treatment  with  results  being  compared  to  those  of  two  additional  treatments  containing  presentation  interference. 
The  independent  variable  was  treatment  (presentation  version).  The  dependent  variables  were  (a)  the  proportion  of 
correct  responses  to  the  follow-up  test  and  (b)  responses  to  the  belief  questionnaire.  Data  were  analyzed  using 
analysis  of  variance  (ANOVA).  Fisher's  Least  Significant  Difference  (LSD)  post  hoc  analysis,  and  calculation  of 
effect  size. 


Results 

Hypothesis  1 

A presentation  that  is  intentionally  presentation  interference  free  will  result  in  higher  posttest  seines  than 
will  presentations  with  interference.  Treatment  1 did  result  in  a higher  mean  score  on  the  folUnv-up  lust  than 
Treatments  2 and  3.  An  analysis  of  variance  (Table  1 ) shows  that  a significant  difference  in  po.sliest  scores  exists  hy 
treatment,  F(2,  1 17)  = 4.575,  p = .01 2. 


Table  /.  Analysis  of  Variance  of  Posttest  Scores  by  Treatment 


SOURCE  SS  DF  MS  F 

TREATMENT  0.145  2 0.072  4.575  * 

ERROR  1.822  115  0.016 


*£=.01 


Fisher's  Least  Significant  Difference  reveals  that  the  difference  between  that  the  posttest  means  of 
Treatments  1 and  2 (.083)  is  significant  (p  = .004),  with  a moderate  effect  size  of  .55.  The  difference  in  posttest 
means  between  treatments  1 and  3 (.064)  is  also  significant  (p  = .027)  with  a moderate  effect  size  of  .43. 


Hypothesis  2 

As  the  amount  of  presentation  interference  in  a presentation  increases,  corresponding  posttest  scores  will 
decrease.  Group  means  by  treatment  arc  displayed  in  Table  2.  As  expected.  Treatment  2,  with  two  types  of 
interference,  resulted  in  lower  posttest  scores  than  did  Treatment  1,  with  no  intentional  interference.  However, 
contrary  to  the  hypothesis,  Treatment  3,  with  the  most  presentation  interference,  did  not  result  in  a significantly 
lower  posttest  scores  than  Treatment  2. 


Table  2.  Posttest  Means  by  Treatment  Group:  Percent  Correct 


Learning  Cost 
N 


.86 


37 


Treatment 

2 

j .80 

41 


.82 

40 


Hypothesis  3 

Subjects  viewing  the  interference  free  presentation  will  report  more  positive  beliefs  regarding  message 
content  and  deliver , method  than  will  those  viewing  presentations  containing  presentation  interference . All  subjects 
completed  a 1 2- item  belief  questionnaire.  Responses  were  collected  via  a seven-point  bipolar  probability  scale.  For 
items  1 - 10,  7 - "Strongly  Agree"  and  1 = "Strongly  Disagree;"  for  item  11,7  = "Extremely  Concerned"  and  1 = 
"Not  at  all  Concerned;"  and  for  item  12,7  = "Extremely  Often"  and  1 = "Not  at  All."  It  was  hypothesized  that 
subjects  in  the  control  group  (Treatment  1)  would  respond  more  positively  to  the  items  on  the  questionnaire  than 
would  subjects  in  groups  two  or  three.  Table  4 presents  the  questionnaire  items  and  mean  responses. 
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Table  3.  Mean  Responses  to  Belief  Questionnaire 


Item 

I thought  the  presentation  was  believable  and  authoritative. 

I learned  a lot  from  this  presentation. 

I thought  the  presentation  was  too  fast. 

I thought  the  presentation  was  too  slow, 

I thought  the  presentation  was  paced  just  about  right. 

I expect  some  of  my  behaviors  to  change  as  a result  of  viewing  this  presentation. 
In  general,  computer-generated  presentations  arc  very  beneficial. 


I trust  and  respect  teachers  who  use  compuLcr  presentations  in  the  classroom  more 
than  I trust  and  respect  those  who  do  not. 

10. 1 have  had  lots  of  experience  with  screen  design  and  multi-media  production. 

1 la.  Before  viewing  the  presentation,  I was  concerned  about  iny  getting  skin  cancer. 

1 lb.  Now,  after  viewing  the  presentation,  lam  concerned  about  my  getting  skin  cancer. 
12a.  Before  I viewed  the  presentation,  I applied  sunscreen  this  often: 

12b.  After  viewing  the  presentation,  I expect  to  apply  sunscreen  this  often: 


T! 

T2 

T3 

F 

6.76 

6.46 

6.60 

1.76 

5.70 

5.49 

5.68 

.47 

2.27 

2.34 

2.23 

.07 

2.87 

3.07 

3.20 

.41 

5.46 

5.42 

5.38 

.02 

4.60 

4.27 

4.75 

.79 

6.16 

5.66 

6.40 

4.92 

.25 

3.98 

4.65 

1.57 

3.16 

2.61 

3.50 

2.86 

2.54 

2.22 

2.68 

.73 

3.38 

3.60 

3.46 

.18 

4.97 

5.24 

5.13 

.31 

3.08 

3.17 

3.10 

.03 

5.00 

4.78 

5,05 

.32 

Subjects  in  all  groups  thought  the  presentation  was  believable  and  authoritative,  with  mean  responses  for 
item  1 ranging  from  6.46  to  6.76  of  7.00  possible.  Responses  across  all  groups  were  also  positive  for  item  2,  “I 
learned  a lot  from  this  presentation,”  with  mean  responses  ranging  from  5AC  to  5.7  of  7.00,  and  for  item  5,  “I 
thought  the  presentation  was  paced  just  about  right,”  with  mean  responses  ranging  from  5.38  to  5.46  of  7.00. 

For  items  6,  “I  expect  some  of  my  behaviors  to  change  as  a result  of  viewing  this  presentation,”  and  8. 
“Instructors  who  use  computer  presentations  arc  usually  better  than  those  who  do  not,”  responses  were  uniformly 
neutral  across  treatment  groups.  For  item  9:  “1  trust  and  respect  teachers  who  use  computer  presentations  in  the 
classroom  more  than  I trust  and  respect  those  who  do  not,”  responses  were  slightly  negative  and  the  difference 
between  treatment  groups  was  not  significant  (p  = .06).  Responses  to  item  10,  ‘‘I  have  had  lots  of  experience  with 
screen  design  and  multimedia  production,”  were  uniformly  low  across  treatment  groups  (2.57,  2.22,  and  2-68, 
respectively,  on  a scale  of  one  through  seven  with  one  being  “strongly  disagree”). 

Analysis  of  variance  revealed  that  the  difference  in  mean  treatment  group  responses  to  item  7:  "In  general, 
computer-generated  presentations  are  very  beneficial”  was  significant  (p  < .01),  as  shown  in  Table  4.  This  item 
directly  addresses  one  of  the  primary  questions  of  this  study.  Means  by  treatment  for  item  7 arc  shown  in  Tabic  5. 
Group  3,  which  received  the  treatment  with  the  most  types  of  interference,  had  the  highest  mean  response  to  this 
item  (6.4),  followed  by  groups  1 and  2 (6.2  and  5.7,  respectively). 

Because  significance  was  found  using  ANOVA,  LSD  post  hoc  tests  were  performed  amongst  all  treatment 
means.  The  difference  between  treatment  groups  1 and  2 regarding  item  7 was  significant  (p  = .043)  with  a low 
effect  size  of  .35.  The  difference  between  treatment  groups  2 and  3 was  also  significant  (p  = .003),  with  a moderate 
effect  size  of  .52.  There  was  no  significant  difference  in  mean  response  between  Treatments  1 and  3 (p  .339). 


Table  4 . Analysis  of  Variance  of  Iron  7,  Computer  Presentations  Beneficial 


SOURCE 

SS 

DF 

MS 

F 

TREATMENT 

11.61 1 

2 

5.806 

4.915  * 

ERROR 

135.847 

115 

1.181 

* p < .01 
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Table  5.  Treatment  Means  for  Item  7,  "Computer-generated  presentations  are  beneficial 


Agreement 

N 


6.2 


37 


Treatment 

2 

41 


6.4 


40 


Responses  to  items  1 1 and  12  did  not  vary  significantly  by  treatment,  nor  was  there  any  significant 
difference  between  treatments  regarding  the  gain  from  item  1 la  to  1 lb  or  from  item  12a  to  12b.  Note  that  items  ! 1 
and  12  did  not  deal  directly  with  presentation-related  issues,  rather  they  focused  on  individuals’  concerns  and  habits 
regarding  their  own  health  and  skin  care  habits. 

Three  hypotheses  were  investigated  in  this  study.  The  first,  whether  Treatment  1 (no  intentional 
interference)  would  result  in  a higher  mean  posttesl  score  than  either  Treatment  2 or  Treatment  3,  is  supported  by 
the  results.  The  difference  between  Treatments  1 and  2 was  significant  (p  < .01 ).  The  difference  between  Treatments 
1 and  3 was  also  significant  (p.  = .02).  The  second  hypothesis,  that  increases  in  the  amount  of  interference  would 
result  in  lower  posttest  scores,  was  not  supported;  There  was  no  significant  difference  in  learning  cost  between 
Treatments  2 and  3. 

The  third  hypothesis,  that  the  presentation  with  no  intentional  interference  would  result  in  more  positive 
responses  to  the  belief  questionnaire  are  supported  in  part.  Treatment  I did  result  in  a significantly  more  positive 
mean  response  to  item  7 ("In  general,  computer  generated  presentations  are  very  beneficial")  than  did  Treatment  2, 
as  expected.  However,  Treatment  3,  with  the  most  intentional  interference  and  which  was  expected  to  result  in  less 
positive  responses  than  cither  Treatments  1 or  2,  also  resulted  in  a significantly  more  positive  mean  response  than 
Treatment  2.  There  was  no  significant  difference  between  Treatments  1 and  3 regarding  item  7. 

Discussion 

Presentation  Interference 
Interference  Versus  No  Interference 

Hypothesis  1 predicted  that  a presentation  without  intentional  interference  would  result  in  higher  posttest 
scores  than  would  those  containing  presentation  interference.  Comparing  group  means  for  the  interference  free 
presentation  with  each  of  the  two  interference  presentations  revealed  that  the  posttest  scores  for  the  interference  free 
presentation  group  were  significantly  higher  than  those  of  cither  of  the  two  groups  receiving  presentations 
containing  interference.  This  is  not  surprising  in  light  of  existing  single  element  research  and  studies  that 
unintentionally  included  interference  (c.g.,  Dwyer,  1978).  Unlike  Grabinger  and  Albers'  (1988,  cited  in  Grabingcr 
1989)  study,  subjects  in  this  study  did  not  have  control  over  the  amount  of  time  allowed  for  each  screen,  a pacing 
condition  closer  to  that  in  which  an  instructor  lectures  with  a PowerPoint  presentation,  and  controls  the  pace. 
Subjects'  responses  to  pace-related  questionnaire  items  indicate  that  the  pacing  in  the  present  study  was  adequate  for 
all  three  treatment  groups. 

Interference  Versus  Mere  Interference. 

Hypothesis  2 predicted  that  increasing  the  amounts  of  presentation  interference  would  result  in  lower 
posttest  scores.  This  hypothesis  is  partially  supported.  Between  groups  1 and  2,  the  significant  difference  (p  =.004) 
in  posttest  scores  is  attributed  to  the  presence  of  presentation  interference.  Treatment  2 contained  two  intentional 
changes  from  Treatment  1 . The  background  color  changed  from  blue  to  pink,  and  the  font  changed  from  a basic  san 
serif  font  to  a more  ornate  display  font  that  appeared  to  be  smaller.  Two  types  of  interference  were  intentionally 
introduced,  yet  the  net  result  was  at  least  three  types  of  interference  because  the  visual  size  of,  for  example,  "48 
points,"  is  not  identical  from  font  to  font.  The  apparent  size  difference  from  identical  point  sizes  supports  Mukhcrjcc 
and  Edmonds  (1994)  point  that  the  whole  is  greater  than  the  sum  of  the  parts,  and  illustrates  why  a prescriptive  that 
computer  delivered  text  should  he  “at  least  26  points''  (Gibson  & Mayta,  1992,  cited  in  Knupfcr,  1994)  can  be 
meaningless  or  misleading.  Past  single  clement  research  attributed  differences  in  learning  to  the  change  of  a single 
clement.  While  manipulation  of  only  one  clement  makes  cause  and  effect  easier  to  ascribe,  the  value  to  actual 
practice  is  limited  because  screen  designs  are  made  up  of  multiple  elements  that  often  have  more  than  a single  flaw. 
As  the  present  multiple  clement  study  demonstrated  (c.g.,  point  size  versus  apparent  size),  single  intentional  element 
changes  can  have  multiple  apparent  effects.  The  difference  between  treatment  groups  in  the  present  study  is  not 
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entirely  attributable  to  any  single  element  (c.g.,  background  color)  because  the  purpose  and  design  of  the  study 
required  multiple  element  manipulations.  The  focus  of  this  study  was  not  to  determine  which  kinds  of  interference 
arc  worse  than  others,  but  to  determine  whether  the  presence  of  interference  in  realistic  combinations  had  a 
significant  effect  on  learning  from  a computer  generated  presentation.  The  study  also  sought  to  determine  whether 
increasing  the  interference  present  in  a presentation  would  increase  the  difficulty  in  learning. 

There  was  no  significant  difference  between  Treatment  2 with  three  types  (including  the  apparent  text  size) 
of  interference  and  Treatment  3,  with  five  types  (including  the  apparent  text  size)  of  interference.  This  may  be 
partially  explained  in  terms  of  Thuring,  Hanneman,  and  Haake’s  (1995)  concept  of  “Cognitive  Overhead.” 
Cognitive  Overhead  increases  when  learners  must  become  reoriented  with  inconsistent  design  elements  in  a 
presentation.  For  example,  the  first  intentional  change  in  Treatment  3 from  Treatment  2,  the  minor  misalignment  of 
bullets  and  text,  would  not  be  considered  to  add  to  a learner's  cognitive  overhead  after  the  first  occurrence  because 
their  misplacement  remained  consistent  throughout  the  presentation.  According  to  the  concept,  a presentation  in 
which  the  bullets  and  text  on  alternating  screens  was  aligned  and  misaligned  would  increase  the  cognitive  overhead 
to  a greater  extent  than  would  consistent  misalignment  throughout  an  entire  presentation.  If  the  misalignment  in  this 
study  was  noticed,  it  may  have  become  less  noticeable  or  important  to  learners  as  the  presentation  progressed. 
Reducing  inconsistencies  should  reduce  cognitive  overhead.  In  this  regard,  cognitive  overhead  and  presentation 
interference  arc  similar  in  that  they  both  refer  to  negative  effects  within  specific  contexts  (as  opposed  to  supposed 
context-free  universal  prcscriptivcs). 

The  second  intentional  difference  in  Treatment  3 from  Treatment  2 was  the  screen  transition  sound  effects. 
The  sound  effects  had  no  topical  relevance  to  the  material  being  presented  and  were  expected  to  distract  learners 
from  the  message  content.  However,  results  showed  no  significant  difference  between  Treatment  2 and  Treatment  3. 
Participants,  wearing  headsets,  viewed  the  presentations  via  Power  Macintoshes  in  a busy,  open-use  computer 
laboratory.  During  data  collection  the  lab  was  in  nearly  normal  use.  Althougn  the  computers  not  being  utilized  for 
the  study  had  messages  taped  to  the  monitors  requesting  that  users  work  quietly,  there  were  consistent,  typical  noises 
and  distractions.  All  three  treatments  included  audio  narration  for  each  screen  but  Treatments  1 and  2 were  silent 
during  screen  transitions,  allowing  ambient  noises  and  distractions  to  be  available  to  participants.  Only  Treatment  3 
had  sound  effects  and,  although  they  were  intended  as  a distraction,  they  may  have  minimized  the  distractions  from 
others  working  in  the  crowded  computer  laboratory  and  kept  learners*  attention  on  the  presentation.  There  is  no 
evidence  from  this  study  that  the  sound  effects  either  aided  or  hindered  learning. 

Results  suggest  that  intentionally  increasing  interference  docs  not  necessarily  lead  to  lower  posttest  scores. 
Further,  elements  that  can  be  reasonably  expected  to  result  in  interference  may,  in  some  contexts  and  settings,  have 
positive  or  neutral  effects.  In  a particular  setting  and  context,  there  may  be  a threshold  at  which  additional 
interference  results  in  compensation  by  learners.  The  phenomena  of  higher  achievement  from  what  could  be 
reasonably  expected  to  be  more  difficult  screens  is  evidenced  in  the  literature  (c.g.,  Knuplcr  and  Mclsaac,  1992). 
Cermak  and  Craik  (1979,  cited  in  Pettersson,  1993)  found  that  when  learners  perceive  a task  as  more  demanding, 
they  process  the  materials  more  deeply  and  better  remember  both  details  and  main  ideas.  This  is  also  suggested  by 
research  conducted  by  Grabinger  (1989).  However,  busier  or  more  elaborate  displays  do  not  necessarily  improve 
learning.  According  to  Hannafm  and  Hooper  (1989), 

Although  the  student  may  be  more  aroused  by  visually  stimulating  displays,  and  important  details  may  be 
illuminated  by  improving  legibility,  such  displays  may  also  increase  the  processing  burden  on  the  learner 
and  cause  shallow  processing  of  important  lesson  content,  (p.  156) 

Beliefs  Regarding  Computer  Generated  Presentations 

Hypothesis  3 slated  that  subjects  viewing  the  interference  free  presentation  would  respond  more  positively 
to  belief  statements  regarding  message  content  and  delivery  method  than  would  those  viewing  presentations 
containing  presentation  interference. 

Overall,  subjects  responded  positively  to  all  three  versions  of  the  presentation.  Tile  only  significant 
differences  in  subject  responses  to  the  belief  questionnaire  were  to  item  7,  “In  general,  computer  generated 
presentations  arc  very  beneficial.”  Subjects  from  all  three  groups  agreed  with  this  statement,  however  the  responses 
from  group  1 (with  no  intentional  interference)  and  group  3 (with  the  most  intentional  interference)  were 
significantly  higher  (p  = .043,  p s=  .003,  respectively)  than  those  from  group  2.  The  significant  difference  between 
Treatments  1 and  2 was  expected  but  the  fact  that  Treatment  3 resulted  in  a higher  mean  response  than  Treatment  2 


This  may  he  explained  in  light  of  findings  from  a survey  reported  by  Sammons  (1995)  of  500  students  in  15 
university  classes  in  which  computer  generated  presentations  were  used.  Results  from  that  survey  indicated  that 
although  students  generally  liked  the  use  of  computer  aided  presentations  in  classrooms,  according  to  Sammons, 
many  students  thought  that  the  computer  was  not  being  used  to  its  advantage  and  that  “using  a computer  as  a 
glorified  overhead  projector  is  a waste”  (p.  68).  Common  comments  regarding  the  screen  designs  of  instructor 
developed  presentations  were  the  following: 

1 . Letters  arc  often  not  large  enough  or  legible. 

2.  There  is  not  enough  color  contrast. 

3.  Too  much  information  appears  on  each  screen.  Students  suggest  including  only  key  points. 

4.  Students  prefer  an  outline  format  with  a hierarchy  of  points  and  subpoints  rather  than  bullets  all  at  one 

level,  (p.  68) 

Considering  Sammons’  study  in  which  subjects  believed  that  classroom  presentations  did  not  use  the 
computer  to  its  advantage,  one  might  infer  that  subjects  in  group  3 (with  transitional  sound  effects)  may  have 
perceived  that  the  presentation  took  advantage  of  the  computer’s  options.  The  difference  between  treatment  groups  1 
(no  interference)  and  3 (the  most  interference)  regarding  item  7 was  not  significant.  Subjects  in  group  1 felt  strongly 
that  presentations  are  beneficial.  That  belief  was  supported  by  their  performance  on  the  posttest.  Subjects  in  group  3 
believed  as  strongly  as  those  in  group  1 that  presentations  arc  beneficial,  but  they  performed  significantly  worse  on 
the  posttest  than  did  subjects  in  group  1.  Treatment  3 did  result  in  positive  beliefs  about  the  medium  but  did  not 
result  in  improved  learning.  Results  indicate  that  although  subjects  who  view  well  designed  presentations  and  those 
who  view  presentations  containing  interference  may  both  report  strong  beliefs  regarding  the  beneficial  nature  of 
computer  generated  presentations,  presentations  containing  interference  significantly  reduce  learning. 

Subjects  generally  agreed  with  item  8,  “Instructors  who  use  computer  presentations  are  usually  belter  than 
those  who  do  not,”  with  neutral  means  (3.98-4.65  out  of  7)  across  groups  and  no  significant  difference  between 
treatment  groups.  Responses  to  item  9,  “I  trust  and  respect  teachers  who  use  computer  presentations  in  the  classroom 
more  than  I trust  and  respect  those  who  do  not,''  were  slightly  negative  for  all  groups  (2.61-3.5  out  of  7). 

Implications  of  the  Study 

Presentation  Interference  is  a viable  construct  for  use  in  screen  design  development  and  research.  Previous 
research  on  screen  design  was  driven  by  the  search  for  identifiable  individual  element  prescriptives  to  be  used  in  an 
additive  process.  Research  has  shown  that  manipulating  single  elements  can  affect  learning  (c.g.,  Aspillaga.  1991. 
Winn  & Solomon,  1993).  However,  presentations  do  not  consist  of  only  a single  element  and  the  combination  of 
single  elements  results  in  a new  whole.  Moreover,  a set  of  specific  universal  visual  design  formulas  cannot  possibly 
be  appropriate  to  every  context.  Design  prescriptives  derived  from  single-element  research  arc  a useful  and  logical 
starting  point  but  several  researchers  (c.g.,  Grabingcr,  1989;  Hannafin  & Hooper,  1989:  Haag  & Snetsigner,  1994; 
Mukherjee  & Edmonds,  1994)  agree  that  they  are  not  enough.  In  addition,  presentation  development  should  include 
cognitive  strategics  (c.g.,  Hannafin  and  Hooper,  1989),  and  a followup  evaluation  process  to  eliminate  presentation 
interference.  Looking  for  and  eliminating  interference  is  always  context-based. 

This  study  supports  the  assertion  that  screen  design  Haws,  in  this  ease,  relatively  conservative  ones,  can 
negatively  affect  learning.  The  purpose  of  this  research  was  not  to  quantify  or  categorize  interference  in  order  to 
devise  universal  rules  regarding  the  “right”  way  to  use  elements.  Quantifying  design  choices  is  of  limited  value. 

In  practice,  no  designer  would  knowingly  leave  one  poorly  executed  element  in  because  another  was  “worse” 
according  to  research.  A label  such  as  “inappropriate  design  choice”  is  only  meaningful  in  relation  to  the  particular 
context  in  which  the  choice  is  inappropriate.  The  goal  is  not  to  choose  one  type  of  interference  over  another,  but  to 
eliminate  interference  altogether. 

The  concept  of  presentation  interference  is  not  intended  as  a replacement  for  any  of  the  existing  screen 
design  constructs.  The  process  of  identifying  and  eliminating  presentation  interference  is  intended  to  augment  the 
design  process  as  a context  relevant  way  to  evaluate  and  improve  existing  presentations  and  to  evaluate  whether  or 
not  constructs  such  as  comprehensibility  and  readability  have  been  achieved. 

Continued  emphasis  on  single  element  research  will  not  allow  for  prac.ical  exploration  of  the  visual  gestalt. 
Researchers  arc  not  likely  to  gel  beyond  two  or  three  elements  if  they  must  measure  every  single  element  and  every 
multiple  element  combination  by  degrees  to  get  to  the  point  of  realistic  combinations.  The  present  study  looked  at 
interference  created  by  inappropriate  use  of  four  (or  arguably,  five)  elements.  In  order  to  look  at  the  combination  of 
only  four  types  of  interference  and  be  able  to  quantify  and  attribute  the  effects  to  individual  elements  or  specific 
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combinations,  15  treatments  would  have  to  be  developed  Cl  control  + 4 single  + 4 + 3 + 2+1  = 15).  Using  the 
generally  accepted  guideline  of  a minimum  size  of  30  subjects  for  each  experimental  treatment  group  (c.g.,  Gay, 
1992),  at  least  450  subjects  would  be  required.  Conducting  or  replicating  such  a study  is  extremely  impractical  and. 
even  if  it  were  not,  the  use  of  such  quantifiably  categorized  single  elements  would  be  of  strictly  limited  use. 

Strengths  and  Limitations 

This  study  began  with  a contextually  appropriate  arrangement  of  elements  to  present  realistic  content. 
Presentation  design  decisions  were  context  based.  Expert  development  and  feedback  regarding  the  design  of  the 
presentation  were  used  to  establish  the  interference  free  control  treatment.  The  interference  free  control  treatment 
was  measured  against  treatments  with  realistic  combinations  of  interference.  This  allowed  a more  meaningful  result 
than  would  comparing  two  treatments  that  both  reflected  realistic  practice.  The  study  used  a holistic  approach  and 
acknowledged  the  visual  gestalt  by  including  multiple  Hawed  elements,  in  realistic  combinations,  in  a presentation 
with  real  content.  Replication  could  be  expected  to  find  similar,  general  kinds  of  results  but  not  identical  findings 
because  exact  contexts  will  not  be  duplicated. 

Limitations  of  this  study  that  may  affect  validity  or  include  the  age  of  subjects  and  their  voluntary 
participation,  although  the  effects  of  the  latter  factor  were  reduced  by  randomization.  Students  in  this  study 
participated  individually  with  headsets.  In  a crowded  lecture  hall,  sitting  with  friends  and  fellow  students,  sound 
effects  may  have  an  entirely  different  effect. 

Recommendations  for  Future  Research 

Several  suggestions  from  this  study  arc  deserving  of  further  research; 

Replicate  this  study  with  these  and  other  types  of  interference  in  presentations  based  on  different  content 
and  contexts  but  with  similarly  rigorously  prepared  interference 

free  control  versions.  The  importance  of  beginning  with  a contextually  based  ideal,  which  can  then  be  measured 
against  realistic  practice  cannot  be  overstated.  The  context  based  interference  free  control  treatment  can  be  measured 
against  presentations  based  on  the  control  version  to  which  combinations  of  presentation  interference  have  been 
cumulatively  increased  (e.g.,  three,  five,  and  seven  intentionally  added  examples  of  interference). 

Conduct  research  to  investigate  whether  at  some  point,  the  addition  of  more  interference  docs  not  increase 
learning  cost.  This  should  be  investigated,  not  in  an  effort  to  quantify  or  categorize  types  of  interference,  hut  to  see  if 
learners  can  compensate  for  extra  interference  with  extra  effort.  If  such  a point  exists  in  replication,  context  dictates 
that  point  will  necessarily  be  different  for  every  study. 

Revise  the  belief  questionnaire  to  include  items  such  as  “Learning  from  this  presentation  was  easy"  or 
“This  presentation  required  concentration.” 

Conduct  research  with  similar  treatments  but  expose  each  entire  treatment  group  to  the  corresponding 
presentation  at  one  time  in  a lecture  hall.  While  this  may  be  difficult  practically,  the  method  may  reve.il  different 
responses  to  transitional  sound  effects  than  resulted  from  individual  participation  with  headphones. 

Consider  adding  a later  followup  test  to  measure  whether  there  is  a difference  in  recall  over  time. 
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Wait  ‘till  ya  hear  this  one:  Professional  Development  Through  Anecdotes 


Katherine  S.  Cennamo 
Virginia  Tech 


This  paper  is  part  of  a larger  their  practice  of  teaching  mathematics  to  be  consistent  with  reform  initiatives, 
and  2)  investigating  a model  of  professional  development  that  addresses  problematic  issues  for  the  teachers.  For  us, 
as  instructional  designers  investigation  that  has  the  dual  goals  of:  1)  determining  problematic  issues  for  teachers  as 
they  change  and  developers,  the  problematic  issues  addressed  though  the  professional  development  process  are  not 
as  important  as  how  the  teachers’  knowledge  is  generated  and  shared;  thus  I will  focus  on  the  second  goal  of  our 
investigation.  This  paper  will  describe  the  evolution  of  our  model  of  professional  development  as  we  explored 
techniques  to  incorporate  technology  to  provide  social  support  for  changed  practice. 

Nature  of  Professional  development 

The  professional  development  of  practicing  professionals  often  involves  modifying  or  expanding  upon 
deeply  seated  ideas  and  beliefs  about  the  way  their  business  should  be  conducted.  Teachers  embark  on  the  mission 
of  teaching  with  implicit  and  explicit  beliefs  about  the  way  classrooms  should  be  managed  and  content  should  be 
taught.  These  beliefs  may  be  implicit  or  explicit,  but  they  arc  the  summation  of  < variety  of  factors  including  prior 
experiences  with  their  own  teachers,  knowledge  gained  from  courscwork  or  independent  reading,  and  perceptions  of 
themselves  in  the  role  of  trachcr  (Kagan, 1992).  As  such,  the  professio  n!  development  of  individuals  with 
experience  in  a given  area  of  practice  may  require  that  they  rcconceptualize  their  professional  roles  and 
responsibilities  (Goldsmith  & Schifter,  1993). 

Most  attempts  to  change  the  practice  of  experienced  teachers  have  consisted  of  experts  training  teachers  in 
new  practices  (Feldman,  1996).  “While  these  types  of  inservice  education  can  be  effective,  they  make  little  use  of 
the  expertise  of  the  teachers  being  inserviced.”  (Feldman,  1996,  p.  513).  The  challenge  for  me  as  an  instructional 
designer  was  to  develop  teacher  education  activities  (hat  worked  with  the  teacher,  rather  than  worked  on  the  teacher. 
Our  intent  was  to  create  "instruction"  that  would  provide  opportunities  for  teachers  to  identify  issues  that  were 
relevant  to  them  and  to  enhance  their  practice  in  areas  that  were  personally  meaningful  to  them. 

In  addition,  we  wanted  our  approach  to  professional  development  to  be  consistent  with  the  theoretical  basis 
of  their  classroom  curriculum.  Based  on  a Piagctian  approach  to  learning,  conceptual  change  occurs  through 
experiences  that  induce  cognitive  conflict  and  indicate  inadequacies  in  current  thinking  (Piaget,  1985).  Learners 
should  be  actively  involved  in  trying  things  out  to  see  what  happens,  posing  their  own  questions  and  seeking  their 
own  answers,  reconciling  what  was  found  at  one  time  with  what  was  found  at  another,  and  comparing  their  findings 
with  those  of  others  (Driscoll,  1994).  Through  the  processes  of  assimilation  and  accommodation  teachers  may 
reconccptualize  their  roles  as  teachers  (Goldsmith  & Schifter,  1993).  Through  assimilation,  new  ideas  are 
incorporated  into  existing  schemes;  through  accommodation,  existing  ideas  arc  modified  to  account  for  new 
experiences.  New  ideas  must  be  consistent  with  learner’s  experience  in  order  to  be  acceptable  to  the  learner; 
modified  ideas  must  account  of  all  new  and  previous  data.  In  addition,  interactions  with  peers  axe  an  important 
source  of  cognitive  development;  through  social  interactions,  as  well  as  through  action,  learners  can  become 
dissatisfied  with  their  existing  knowledge.  Through  comparing  their  ideas  with  the  ideas  of  others,  learners  may 
find  new  ideas  that  arc  plausible  and  useful  alternatives  to  their  current  conceptions  (Goldsmith  & Schifter,  1993). 
We  wanted  to  create  opportunities  for  teachers  to  express  and  examine  their  beliefs,  values,  and  practice,  and  to 
compare  those  personal  beliefs,  values,  and  practices  to  those  of  others. 

In  addition,  many  skills  needed  by  these  professionals  arc  “ill-structured”  (Jonasscn,  1997).  Ill -structured 
problems  arc  those  for  which  multiple  opinions  and  perspectives  exist;  in  order  to  find  a solution,  reflective 
judgments  must  be  made  about  the  advantages  and  disadvantages  oi  a particular  solution  for  different  people  and 
contexts.  Often  there  is  no  “right  answer”  to  dilemmas  they  may  face,  only  a collection  of  possible  solutions  or 
options. 

Cases  for  professional  development 

Case-based  instruction  can  provide  opportunities  for  conceptual  change  to  occur  and  meet  the  conditions 
for  teaching  in  ill-structured  knowledge  domains  (Driscoll,  1994;  Jonasscn,  1997).  Cases  provide  a context  through 
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which  Icarncrs  can  explore  dilemmas  and  solutions.  They  can  be  a description  of  a real  event,  a modification  of  a 
real  event,  or  a fictional  event.  Typically,  a case  discussion  begins  with  the  presentation  of  the  ease,  a discussion  of 
what  is  know  and  not  know  (relevant  facts  of  the  case),  and  discussion  of  possible  interpretations  and  solutions  to 
the  case  dilemma. 

They  arc  particularly  useful  for  novices  in  a content  area  (prcscrvicc  teachers,  for  example)  in  that  they 
provide  a point  of  reference.  Often  preservice  teachers  arc  unable  to  imagine  what  it  would  be  like  to  be  a teacher. 
They  lack  a frame  of  reference  to  which  to  apply  the  new  knowledge  gained  in  courscwork.  They  emerge  from  their 
education  with  an  idealized  view  of  leaching.  And  when  faced  with  the  complexity  of  the  classroom  envronmeni, 
may  abandon  what  they  learned  and  instead,  react  by  teaching  in  the  way  they  were  taught.  They  may  be  unable  to 
resolve  conflicts  between  what  they  have  learned  about  teaching  in  their  university  classes  and  what  they  know 
about  teaching  from  their  prior  experience,  in  face  of  the  realities  of  the  classroom. 

Experienced  teachers  enrolled  in  professional  development  activities,  on  the  other  hand,  already  have  a 
current  frame  of  reference  in  which  to  consolidate  their  new  knowledge.  They  arc  acutely  aware  of  the  realities  of 
the  classroom.  We  wanted  to  find  a way  for  practicing  teachers  to  explore  new  ideas  about  teaching  consistent  with 
reform  initiatives  and  consolidate  new  ideas  with  their  knowledge  gained  through  practice  and  the  “apprenticeship 
of  observation”  (Lortie,  1975).  In  addition,  we  wanted  to  provide  social  support  for  changed  practice  in  an  arena 
where  teachers  could  capitalize  on  their  own  experiences  and  those  of  their  peers  to  explore  issues  relevant  to 
reform-based  teaching  of  mathematics.  And  finally,  we  wanted  to  provide  an  opportunity  for  them  to  extend  their 
modified  ideas  to  their  classrooms. 

Anecdotes  for  Professional  Development 

Similar  to  cases,  anecdotes  of  teaching  episodes  have  provided  a means  for  teachers  to  work  on  their 
practice  in  a variety  of  content  areas  (Feldman,  1996;  Joworski,  1989;  Mathematical  Association,  1991;  Stock, 
1993).  An  anecdote  is  “a  narrative  of  i detached  incident,  or  of  a single  event,  told  as  being  in  itself  interesting  or 
striking’1  (Oxford  English  Dictionary,  1971,  as  cited  in  Feldman,  1996).  Like  eases,  anecdotes  situate  “specific 
teaching-learning  moments  in  the  material  circumstances  in  which  they  occur”  (Slock,  1993,  p.  185). 

In  her  discussion  of  the  use  of  anecdotes  to  examine  leaching  and  learning  in  English  classrooms,  Patricia 
Slock  (1993)  proposed  that  teachers  continually  develop  their  understanding  as  they  recreate  the  anecdote  through 
successive  retellings.  “That  teachers  tell  anecdotes  when  discussing  their  leaching  is  common  knowledge.  What  is 
not  generally  recognized  arc  the  functions  that  the  shaping,  reshaping,  and  rehearsing  of  anecdotes  play  in  the 
research  we  informally  and-  I would  claim-  systematically  conduct  into  our  practice”  (Stock.  1993,  p.  IS4).  By 
drawing  attention  to  specific  incidents  in  their  teaching,  teachers  invite  others  to  join  them  in  attempting  to  make 
sense  of  the  incidents  in  order  to  improve  their  practice  (Slock,  1993). 

Allan  Feldman  (1996)  also  determined  that  anecdotes  played  a significant  role  in  teachers’  research  into 
their  own  practice.  He  convened  a group  of  eight  physics  teachers  to  engage  in  collaborative  action  research  on 
their  practice.  The  group  meet  for  one  and  a half  hours  approximately  every  three  weeks  for  two  years.  Feldman 
audiolaped  the  sessions  to  examine  “the  ways  that  teachers’  knowledge  about  teaching  and  their  educational 
situations  grow  w'hcn  they  arc  engaged  collaborativcly  with  other  teachers  in  inquiry  on  their  own  practice” 
(F'Alman,  1996,  p.  513)  Upon  analysis,  he  found  three  practices  pervading  their  work  together:  anecdotes,  trying 
things  out,  and  systematic  inquiry. 

The  role  of  the  anecdote  was  critical  throughout  all  three  processes.  Often,  teachers  would  relate  an 
anecdote.  Others  would  respond  in  one  of  three  ways:  by  telling  a related  anecdote,  hy  asking  for  details  of  what 
was  described  in  the  anecdote  (who,  what,  where,  how,  and  when),  or  by  asking  questions  that  would  provide  them 
with  a better  understanding  of  the  situation  described  in  the  anecdote  (why  something  occurred).  When  teachers 
decided  to  ‘‘try  something”  new.  they  often  tried  something  that  had  been  described  in  another  teachers*  anecdote. 
When  the  teachers  “tried  something”  new,  they  often  reported  the  incidents  to  the  group  as  anecdotes.  And  when 
teachers  engaged  in  systematic  inquiry,  a narrative  analysis  of  their  classroom  practice  seemed  more  powerful  to 
them  than  following  traditional  scientific  methods.  The  teachers  treated  the  narrative  analyses  and  accompanying 
data  the  same  way  they  treated  the  anecdotes:  through  relaying  other  anecdotes,  through  asking  for  details,  or 
through  questioning  for  meaning. 

Feldman  (1996)  believed  each  of  these  responses  to  an  anecdote  was  of  value  to  the  participants.  When 
teachers  responded  to  an  anecdote  with  another  anecdote,  the  additional  anecdotes  added  context  and  begin  to 
“indicate  to  others  how  their  knowledge  is  related  to  the  situations  of  the  tellers'’  (Feldman,  1996,  p.  522).  When 
they  responded  with  questions  that  asked  for  details,  “they  were  seeking  information  that  would  help  make  the 
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knowledge  and  understandings  presented  in  the  anecdotes  more  useful”  to  their  own  experiences  (Feldman,  1996,  p. 
522).  And  finally,  questions  that  focused  on  the  understandings  that  the  anecdote  tellers  had  of  their  situations 
shifted  the  focus  from  the  specific  details  of  the  anecdotes  to  interpretations  of  the  events  that  were  relayed  in  the 
anecdotes. 

These  stages  in  responding  to  anecdotes  arc  similar  to  the  stages  Barbara  Jaworski  (1989;  Mathematical 
Association,  1991)  has  outlined  for  working  with  anecdotes  for  professional  development.  In  her  workshops  with 
mathematics  teachers,  she  presents  an  anecdote,  either  written  or  on  videotape,  and  has  the  teachers  move  through  a 
structured  approach  of  providing  an  account  of  the  events  in  the  anecdote,  followed  by  accounting  for  the  events. 
In  her  workshops,  groups  of  teachers 

• view  or  read  an  anecdote  (relay  anecdote) 

• construct  an  account  of  what  they  saw  or  read  (clarify  details) 

• try  to  account  for  what  was  seen  or  read  (interpret).  In  accounting  for  their  interpretation  of  the  anecdote, 
they  share  related  episodes  in  their  own  teaching  as  examples  (share  anecdotes). 

The  work  of  these  researchers  and  teacher  educators  indicates  that  anecdotes  can  provide  a concrete 
reference  for  practicing  teachers,  a way  of  envisioning  how  ideas  would  work  in  their  own  classrooms.  In  addition, 
as  a means  of  prof<  ^sional  development,  anecdotes  rely  on  the  expertise  of  the  teachers  and  their  ability  to  construct 
and  share  knowledge  and  understanding.  Thus,  we  chose  to  extend  upon  the  work  of  these  researchers  in  several 
ways.  We  wanted  to  ensure  that  the  teachers  would  extend  the  ideas  examined  during  our  professional  development 
activities  to  their  practice.  In  addition,  we  wanted  to  provide  opportunities  for  the  teachers  to  work  on  their  practice 
over  an  extended  period  of  time. 

In  addition,  we  believed  that  a video-based  professional  development  program  could  support  in-service 
teachers  as  they  experimented  with  new  curricular  and/or  teaching  approaches.  Common  approaches  to  using 
videotape  include  the  use  of  video  to  construct  ease  studies  of  exemplary  teaching  practice.  But  although  video- 
based  demonstrations  of  "expert"  teachers  may  encourage  replication,  it  docs  not  encourage  exploration  of  issues  of 
personal  interest.  We  wanted  to  investigate  ways  that  learning  environments  can  be  designed  to  be  customizable  by 
lechers  and  how  videotapes  can  be  used  to  provide  opportunities  for  teachers  to  link  new  ideas  to  their  own 
practice.  Buiioing  from  our  examination  of  the  use  of  anecdotes  for  professional  development  of  practicing  teachers, 
we  explored  a video-based  approach  to  professional  development  using  anecdotes. 

Context  of  our  investigation 

Large-scale  K-12  mathematics  curriculum  reform  efforts  have  been  underway  lor  several  years  (National 
Council  of  Teachers  of  Mathematics,  1989).  As  curriculum  is  reconccptualized  to  emphasize  project-based 
approaches  to  the  development  of  understanding,  the  role  of  the  teacher  is  changing.  These  changes  require  a skills 
in  facilitating  and  coordinating  learning  experiences  for  individuals  and  groups  of  students;  teachers  need  to  conduct 
active,  productive  discussions  of  mathematical  ideas. 

This  professional  development  program  is  targeted  to  teachers  who  desire  to  recreate  their  practice  in 
manner  consistent  with  mathematics  reform  initiatives.  The  teachers  inay  be  in  their  first  year  of  practice,  or  may 
have  many  years  of  teaching  experiences.  However,  all  are  attempting  to  modify  their  practice  consistent  with 
reform  initiatives.  As  teachers  will  voluntarily  choose  to  develop  their  skills  in  reform  hased  mathematics  teaching, 
we  assume  that  teachers  will  be  convinced  of  the  value  of  giving  students  responsibility  for  their  own  learning  prior 
to  becoming  interested  in  this  professional  development  program.  It  is  possible  that  the  teachers  may  have 
experienced  previous  frustrations  when  leaching  in  this  manner.  However,  we  are  assuming  that  the  teachers  arc 
motivated  to  succeed  and  confident  that  they  could  successfully  develop  their  practice  . Typically,  teachers  place 
high  value  on  the  learning  of  their  students . Thus,  we  believed  the  initial  activities  should  demonstrate  the  value  to 
the  students  of  adapting  alternative  methods  of  teaching  mathematics.  In  addition,  we  assumed  the  teachc.s  would 
like  to  quickly  apply  what  they  learned  to  their  classroom  context. 

As  an  instructional  design  and  development  team,  we  began  our  conversations  with  discussions  of  what 
kinds  of  “changes  in  thinking”  we  wanted  to  see  in  the  teachers.  We  knew  there  were  differences  in  teachers’ 
practice  in  mathematics  class,  even  though  all  the  teachers  all  were  using  the  same  problem-based  math  curriculum  . 
However,  we  were  unsure  of  the  nature  of  these  differences. 

As  we  had  no  answers,  we  moved  to  a discussion  of  how  we  would  know  that  changes  in  thinking  had 
occurred.  We  decided  to  analyze  videotapes  of  classroom  mathematics  lessons  to  identify  differences  in  practice 
among  elementary  mathematics  teachers.  Using  a data  resource  which  consists  of  approximately  200  videotaped 
lessons  collected  over  a period  of  one  year  in  five  second-grade  mathematics  classrooms,  we  developed  a classroom 
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coding  scheme*  coded,  and  interpreted  the  teacher's  and  students’  actions  (Wood,  Turner-Vorbeck,  & Walker, 
1996b). 

Differences  in  teacher’s  practice 

Through  detailed  analysis  of  videotapes  of  classroom  practice,  we  identified  three  dimensions  around 
which  teachers'  praetiec  varies  (Wood,  Turner- Vorbcck,  & Walker,  1996a;  Wood,  Walker,  & Turncr-Vorbcck, 
1996). 

* Although  all  the  teachers  had  their  students  vocalize  their  way  of  thinking  about  mathematics,  the  teachers 
varied  in  the  meaning  they  hold  for  "children’s  thinking”.  Some  teachers  had  students  report  their  solutions  to  the 
problem,  others  required  them  to  tell  how  they  solved  the  problems,  and  others  asked  students  to  tell  why  they  did 
what  they  did  . 

* Teachers  also  varied  in  the  extent  to  which  they  allowed  disagreement:  some  set  up  exchanges  that  were 
primarily  between  teacher  and  student;  others  established  exchanges  between  students  and  the  rest  of  the  class. 

* We  also  discovered  that  when  children  shared  their  ideas,  teachers  varied  in  their  expectations  for  active 
participation  by  the  "‘listener”  and  “explainer”. 

From  this  analysis,  we  developed  a theoretical  framework  that  clarifies  issues  that  may  be  important  to 
teachers  as  they  develop  their  mathematics  teaching.  We  identified  three  types  of  context  created  in  classrooms  that 
iniluencc  students’  opportunities  to  c igage  in  mathematical  thinking  and  reasoning;  reporting  of  solutions,  inquiry 
into  the  solutions,  and  argument  about  the  solutions.  These  contexts  arc  a result  of  ways  in  which  teachers  establish 
with  the  hildren  the  social  norms  or  expectations  for  behavior  in  mathematics  class.  As  teachers  and  their  students 
move  vertically  throughout  the  model,  students  move  from  reflective  thinking  to  critical  reasoning;  as  they  move 
horizontally,  students  lake  greater  responsibility  for  learning.  Having  identified  issues  around  which  practices  vary, 
our  challenge  was  to  develop  activili • s that  allow  the  teacher  to  explore  these  issues  in  their  classrooms  in 
personally  meaningful  ways. 

Development  of  the  plan 

Simultaneously,  with  our  analysis  of  classroom  videotapes,  we  begin  to  develop  ou.  x'ay  of  working  with 
teachers.  Using  the  work  of  Barbara  Jaworski  and  the  Open  Univcrsily(  1989;  Mathematical  Association,  1991 ) as  a 
guide,  we  developed  a tentative  way  of  working  with  teachers  to  promote  reflection  on  the  practice  of  mathematics 
leaching. 

In  the  Spring  of  1996,  we  tested  our  way  of  working  with  seven  teachers  who  participated  in  ten  two-hour 
teacher  development  sc  ons.  In  these  sessions,  teachers  selected  video  segments  from  their  own  classrooms  to 
illustrate  issues  of  ini;  .stance  to  them.  Although  we  had  developed  a tentative  pattern  for  discussion,  we 
continually  reflected  on  and  modified  our  way  of  working  throughout  the  sessions  to  determine  the  sequence  and 
questioning  approach  that  was  most  effective  in  generating  discussions  of  multiple  solutions  to  problems  and  issues. 
As  we  conducted  these  sessions,  we  recorded  our  expected  actions  for  the  session  prior  to  the  session,  videotaped 
the  sessions,  recorded  field  notes,  and  made  reflective  notes  following  the  session.  The  videotapes  were  logged  for 
analysis.  The  participants  also  recorded  their  reflections  and  assignments  in  a notebook.  We  analyzed  these  data  to 
identify  patterns  of  behaviors  and  responses,  to  determine  if  the  issue*  teachers  identified  as  important  in  their 
classrooms  were  similar  to  the  issues  we  identified  as  important  based  on  our  analysis  of  classroom  videotapes,  and 
to  refine  our  way  of  working  with  the  teachers.  After  our  analysis  revealed  that  the  problematic  issues  that  arose 
from  the  teacher  sessions  were  consistent  with  those  we  had  identified  from  our  analysis  of  classroom  videotapes, 
we  felt  confident  in  developing  video  segments  to  fccus  discussions  on  identified  issues. 


The  plan 

From  our  analysis,  we  developed  a set  of  videotapes  and  supporting  leader’s  guide  for  professional 
development  of  elementary  mathematics  teachers.  The  materials  consist  of  a series  of  five  videotape  segments  of 
mathematical  discussions  as  they  occurred  in  elementary  classrooms.  The  videotapes  arc  supplemented  with  a 
“leader’s  guide”  that  outlines  our  way  of  working  with  anecdotes.  The  leader’s  guide  includes  statements  that 
should  be  made  to  the  participants  and  notes  to  guide  the  discussion  of  the  sessions.  However,  the  teacher  leader  is 
not  the  authority  in  the  session,  only  the  moderator,  who  is  charged  with  moving  the  discussion  forward,  to  moving 
bc\ond  anecdotes  to  identification  of  issues  and  plans  for  action.  'Flic  loaders  role  is  one  of  chairing  the  session.  He 


or  she  should  ensure  thaL  all  participants  get  a chance  to  voice  their  opinions  and  thoughts  and  should  mediate  so 
that  all  may  have  an  opportunity  to  do  so.  The  leader’s  guide  contains  video  notes,  which  indicate  possible  issues 
that  may  arise  from  the  videotaped  segments,  or  possible  paths  the  discussion  may  take. 

As  planned,  the  professional  development  sessions  we  conducted  with  teachers  were  to  follow  the  steps 
outlined  below: 

1.  Share  an  anecdote.  We  initially  prepared  a set  of  five  videotaped  “anecdotes”.  These  anecdotes  were 
segments  from  actual  elementary  mathematics  classes.  The  segments  were  selected  because  they  illustrated  a 
particular  "block”  in  our  framework.  However,  we  recognize  that  classroom  events  arc  complex;  and  that  each 
segment  could  be  interpreted  from  a different  perspective  than  the  one  for  which  it  was  selected  to  represent.  The 
materials  allow  for  this  flexibility.  The  teacher  working  sessions  begin  by  showing  a videotape  anecdote, 
corresponding  to  one  “block”  in  our  framework. 

2.  Reconstruct  the  details  of  what  was  seen  on  the  videotape.  After  viewing  the  videotape,  individuals 
mentally  reconstruct  the  events  of  the  videotape.  Participants  spend  about  live  minutes  working  in  pairs  to  agree  on 
what  was  seen  on  the  tape,  then  join  together  as  a whole  group  and  spend  about  ten  minutes  discussing  what  was 
seen  on  the  videotape. 

3.  View  the  anecdote  again  and  think  of  related  anecdotes.  Participants  watch  the  videotape  a second  time, 
paying  attention  to  what  the  students  may  be  thinking  as  they  arc  participating.  As  the  group  discusses  the  events 
from  the  perspective  of  the  student,  they  consider  what  the  child  talking  thinks  he  or  she  is  expected  to  do  and  what 
the  children  listening  think  they  arc  expected  to  do.  Participants  reflect  on  how  the  actions  of  the  students  arc 
similar  or  different  from  what  happens  in  their  classroom.  They  arc  asked  to  think  of  one  small  event  from  their 
own  classroom  that  relates  in  some  way  to  the  events  of  the  videotape  and  record  their  reflections  in  their  notebooK. 
The  video  simply  provide  a shared  classroom  experience  from  which  discussions  of  individual  issues  could  begin. 
The  most  important  aspect  of  the  discussions  involves  participants  examining  their  own  situations. 

4.  Distill  or  interpret  the  issues.  In  pairs,  participants  exchange  ideas  of  how  the  discussion  is  similar  or 
different  from  what  they  observe  their  own  students  doing  and  saying  in  their  classroom.  Then  pairs  l.y  to  identify 
the  themes  that  1 : behind  their  experiences.  The  entire  group  joins  together  and  spends  15  or  20  minutes  discussing 
the  theme  or  issues  that  pairs  identified  in  their  discussions.  Genera)  statements  and  interpretations  should  he 
illustrated  with  “for  instances”  from  their  own  experience.  Using  participant's  own  experiences  as  “for  instances” 
grounds  their  statements  in  their  practice,  illustrates  ideas  more  clearly,  and  encourages  participants  to  reflect  on 
their  own  practice. 

5.  Identify  a personal  plan.  Participants  spend  a few  minutes  individually  reflecting  on  specific  action  that 
they  may  want  to  try  out  in  class  to  further  explore  the  issues  they  identified  as  important.  Actions  may  include 
looking  out  for  occasions  when  certain  things  happen,  trying  some  new  approach,  or  making  some  small  changes  in 
behavior.  Then  participants  join  together  as  a group  and  spend  about  ten  minutes  sharing  their  ideas  lor  action. 
Participants  are  directed  to  consider  the  issues  identified  and  possible  action  steps,  and  individually  think  about  their 
expectations  for  future  math  discussions  in  their  classrooms. 

6.  Take  to  practice.  Participants  videotape  one  or  more  class  sessions  to  document  events  in  their 
classrooms.  Before  they  videotape  their  classes,  they  arc  to  list  their  expectations  for  themselves  and  their  students 
in  their  notebook.  After  they  videotape  the  class,  they  are  to  review  the  videotape  and  compare  their  expectations  to 
the  actual  events  that  occurred  in  the  classroom,  recording  their  reflections  in  their  journal. 

7.  Repeat  the  cycle  with  individual  teacher’s  tapes.  When  the  teacher  development  group  reconvenes,  each 
teacher  brings  his  or  her  own  videotape  and  anecdotes  of  classroom  events  to  share  with  the  other  teachers.  The 
process  of  working  with  videotaped  anecdotes  is  repeated  with  individual  teacher’s  tapes  until  the  group  decides  to 
move  to  another  issue. 
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Findings 

We  begin  to  use  our  “way  of  working”  with  a group  of  teachers  in  the  Fall  of  1996.  Through  the  last  18 
months,  we  have  made  the  following  observations: 

Things  don’t  always  proceed  according  to  plan 

Although  we  initially  used  a video  “anecdote”  as  a means  of  initiating  discussion  on  similar  events  in  the 
teachers  classroom,  and  followed  our  way  of  working  with  videotaped  anecdotes,  we  found  that  as  soon  as  teachers 
begin  videotaping  in  their  classrooms,  we  never  returned  to  viewing  the  prepared  tapes.  Teachers  were  enthused 
about  bringing  in  their  own  tapes  to  discuss  with  the  group,  and  wanted  to  continue  in  this  way.  One  issue  lead  to 
another,  and  recognizing  teachers  desire  to  work  on  their  own  classroom,  we  allowed  them  to  continue  in  this  way. 

In  addition,  we  found  that  this  second  group  of  teachers  seemed  to  need  to  focus  on  what  the  students  were 
thinking.  We  recognized  from  the  beginning  that  teachers  were  more  comfortable  focusing  on  the  children’s 
responses  than  focusing  on  the  actions  of  the  teachers,  but  believed  that  they  could  quickly  make  the  connection 
between  what  the  children  were  thinking  and  the  classroom  norms  established  by  the  teacher.  We  found  that  was 
not  true:  they  were  simply  unable  to  judge  what  children  were  thinking  from  their  responses  to  math  problems. 

Thus,  the  focus  of  our  discussions  and  videotapes  that  were  shared  shifted  from  an  examination  of  teacher's 
behavior  to  a focus  on  children’s  mathematical  thinking.  We  realized  that  this  group  of  teachers  lacked  an  essential 
prerequisite  for  focusing  on  the  issues  we  had  identified.  Our  initial  set  of  videotapes  was  created  to  be  a rich 
environment  where  anecdotes  of  classroom  events  could  be  examined  from  many  different  perspectives.  However, 
once  the  focus  of  discussions  shifted  to  an  examination  of  children's  thinking,  we  found  we  had  no  videotapes 
appropriate  for  the  altered  focus.  We  depended  on  teachers  to  bring  in  tapes  of  their  own  to  examine  as  a group. 

With  experience,  stages  in  the  process  merge,  alternating  between  examining  details,  relating  anecdotes,  and 
interpretation. 

Currently  in  the  teacher’s  working  sessions,  each  teacher  shares  her  videotape;  discussions  merge  the 
“reporting  of’  details  with  the  anecdote  telling  and  interpretation  stages.  Although  the  elements  we  included  in  our 
way  of  working  arc  present  in  the  sessions,  they  no  longer  exist  as  formal  concrete  steps.  Instead,  teachers  discuss 
interpretations,  backed  by  details  from  the  tapes.  They  share  instances  where  they  see  their  students  behaving  in 
similar  or  different  fashions  when  appropriate.  Teachers  move  seamlessly  among  the  stages  in  the  process,  in  an 
cyclical,  iterative  manner  rather  than  the  formal  linear  sequence  we  outlined.  The  teacher  educator  has  to  do  much 
less  questioning  and  leading  of  the  discussion  than  initially.  This  way  of  analyzing  and  discussing  video  anecdotes 
has  become  a natural  way  to  view  videotapes. 

Comparing  expectations  with  videotaped  events  as  useful  for  reflection. 

The  use  of  anecdotes,  in  our  case  as  videotaped  examples  of  incidents  from  practice,  can  be  enhanced 
through  the  active  process  of  recording  expectations  prior  to  leaching  a lesson,  leaching  the  lesson,  and  reflecting 
back  on  the  lesson  in  light  of  expectations  for  the  lesson.  Currently,  teachers  videotape  their  students  solving  a 
mathematical  problem  and  vocalizing  what  they  arc  doing  and  why.  Before  they  tape  the  sessions,  they  list  their 
expectations  for  the  children  in  their  journals.  As  they  view  their  videotapes,  they  reflect  on  what  happened.  The 
teachers  share  these  videotapes  in  their  working  sessions  with  the  group.  We  have  found  that  the  tapes  and  journals 
provide  a very  effective  means  of  assessing  what  the  teachers  are  thinking  about  teaching  and  learning  mathematics 
in  their  classroom.  In  addition,  the  theoretical  framework  we  developed  from  our  analysis  of  classroom  videotape:; 
is  useful  means  of  categorizing  teachers'  thinking. 

Teachers  are  reluctant  to  conduct  similar  discussions  on-line 

Currently,  the  teacher  working  sessions  are  devoted  to  examining  what  children  arc  thinking.  However, 
between  teacher  working  sessions,  the  teacher  educator  posts  action  items  to  a web  site  and  teachers  arc  expected  to 
work  on  problematic  issues  that  arise  in  conducting  active  productive  classroom  discussions  on-line.  They  arc  asked 
to  ajrecord  their  expectations  for  their  class  discussion,  b)  conduct  and  videotape  their  class  discussions,  c)  compare 
their  expectations  for  the  class  discussion  with  what  they  observed  on  the  videotape,  and  d)  discuss  their 
expectations  and  what  happened  with  the  other  teachers  in  the  group  though  e-mail  and  listserves.  However,  for  one 
reason  or  another,  the  teachers  have  not  participated  in  these  discussions  on-line. 
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Conclusions 

The  purpose  of  this  presentation  was  to  outline  a “way  of  working”  with  anecdotes  that  channels  our 
natural  tendencies  to  share  our  stories  with  colleagues  into  a strategy  for  the  continued  development  of  practicing 
professionals.  Think  back  to  the  last  time  you  sat  around  a teachers’  lounge  or  met  with  a group  of  professionals 
who  work  primarily  in  isolation.  Sooner  or  later,  they  began  telling  anecdotes  about  the  workplace.  One  anecdote 
leads  to  another.  Participants  share  stories  and  develop  a sense  of  community,  a sense  that  they  arc  not  alone  in  their 
frustrations  and  joys.  But  conversations  of  this  nature  seldom  go  beyond  developing  a sense  of  shared  experiences. 
Although  we  investigated  our  way  of  working  with  elementary  school  teachers  of  mathematics,  this  model  of 
professional  development  can  be  applied  in  a variety  of  contexts  where  the  goal  is  to  empower  professionals  to 
develop  and  improve  their  own  practice. 

This  approach  offers  teachers  a way  of  working  first  as  a group  with  a teacher  educator,  and  later 
individually  as  they  examine  their  own  classroom  context  for  learning.  We  have  provided  support  materials  and 
developed  a professional  development  model  that  encourages  teachers  to  use  video  vignettes  to  focus  on  issues  that 
arise  in  their  classrooms.  In  this  regard,  an  attempt  is  made  to  link  the  work  done  as  a group  to  practice  in 
classrooms  using  video  as  a tool  the  bridge  the  traditional  gap  between  teacher  education  and  classroom  practice. 
The  video  anecdotes  serve  as  a shared  experience  to  generate  discussion  of  issues  in  their  own  classrooms.  By 
working  though  the  steps  in  the  process,  the  group  receives  support  as  they  extend  the  ideas  that  emerge  through  the 
group  discussion  to  practice  in  their  classrooms.  Members  of  the  group  work  together  collectively  10  identify 
issues,  share  experiences,  and  learn  more  about  creating  contexts  in  classrooms  for  learning  mathematics  with 
understanding. 

Currently,  the  project  team  is  analyzing  videotapes  and  journals  from  the  teacher's  working  group  to 
determine  how  teachers  create  and  modify  their  knowledge  through  participating  in  this  type  of  professional 
development  program  and  exploring  ways  to  extend  this  methodology  to  distributed  learning  environments. 
Although  we  have  began  to  explore  the  use  of  anecdotes  for  the  professional  development  of  elementary 
mathematics  teachers,  this  approach  to  professional  development  is  a logical  extension  of  the  ease  method  with 
applicability  to  the  professional  development  of  any  practicing  professional;  1 encourage  others  to  explore  the  use  of 
anecdotes  in  other  contexts  and  with  different  groups. 
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Abstract 

The  Web  has  drawn  more  and  more  distance  educators'  attention  and  been  employed  extensively  as  a 
distance  learning  delivery  tool.  The  key  concept  of  Web-based  distance  education  focuses  on  applying  the  Web  as  a 
bridge  to  provide  instructional  communications  and  activities  between  students  and  teachers.  However , without 
pedagogical  considerations  and  effective  distance  learning  activities,  Web-based  distance  learning  could  be  inactive 
and  dangerous . Therefore , a cybercourse  model  is  designed  and  created  by  infusing  four  beneficial  telecomputing 
activities  into  Web-based  distance  learning  system.  Differences  between  the  regular  system  and  innovative  model 
are  compared  and  contrasted.  The  advantages  and  limits  of  the  model  are  discussed.  Finally,  implications  and 
conclusions  are  provided. 

Introduction 

The  World  Wide  Web  (WWW)  holds  a great  potential  for  distance  education.  The  most  prominent 
characteristics  of  employing  the  Web  as  a distance  learning  tool  is  that  it  becomes  available  not  only  in  classrooms, 
but  dormitory  rooms,  public  library,  and  indeed,  anywhere  in  ihc  world.  As  more  and  more  distance  learning 
activities  occur  on  the  Web,  a comprehensive  wch-bascd  model  with  effective  pedagogical  considerations  is 
relatively  crucial  for  distance  cducalors. 

In  distance  education,  there  is  a high  need  to  vary  both  the  curriculum  materials  and  instructional  activities 
to  engage  students  in  learning.  With  the  Web,  instructors  could  design  and  develop  interactive  curriculum  materials, 
build  interactive  pedagogical  animation  and  sounds  to  motivate  students'  learning,  and  utilize  a variety  of  wch-basoc 
distance  educational  activities  such  as  collaborative  learning,  demonstrations,  interactive  discussion,  and  problem 
solving. 

The  model  presented  in  this  paper  is  not  for  distance  educators  to  follow  instead  it  aims  to  provide  a 
distance  activity  structure  for  them  to  contemplate,  to  apply,  and  then  to  be  able  to  design  and  develop  their  own 
distance  learning  course  which  could  best  meet  their  specific  instructional  needs.  In  this  paper,  hasic  Wcb-hascd 
distance  learning  concept  is  first  synthesized  and  systematized.  Then,  a cybcrcoursc  model  is  presented  by  weaving 
four  benefieial  telecomputing  activities  into  Web-bascd  distance  instruction.  A comparison  between  the  regular 
system  and  new  model  is  followed.  Discussions  on  the  strengths  and  weaknesses  of  the  model  are  made.  Finally, 
implications  and  conclusions  arc  included. 

Web-based  Distance  Education  System 

Utili/.ing  the  Web  as  an  instructional  delivery  system  is  the  basic  concept  of  Wch-bascd  distance  education. 
Applying  WWW  as  a demonstration  tool  is  the  key  instructional  activity  held  by  Wch-hased  distance  educators. 
Instructors  posl  course  syllabus,  curriculum  materials,  and  resources  guidance  on  (he  Web  to  allow  students  to  read, 
learn,  and  conducting  electronic  field  trips. 

In  a Web-based  distance  instructional  system,  teachers  should  prepare,  post,  and  update  their  course 
syllabus,  curriculum  materials  on  the  Web  regularly.  In  the  course  syllabus,  teachers  should  clearly  and  specifically 
addressed  what  type  of  hardware  and  software  configuration  is  needed  for  the  course;  what  instructional  activities 
will  be  conducted  and  how  to  perform  those  activities  during  the  semester;  and  when  the  specific  learning  activities 
will  he  conducted.  How  summativc  and  formative  evaluation  will  be  permeated  into  the  whole  learning  process  also 
have  to  he  considered. 

On  the  other  hand,  students  arc  required  to  access  the  Web  to  review  course  materials,  perform  learning 
activities  and  publish  their  course  products  on  the  Web.  In  the  Wch-bascd  distance  learning  system,  the  Web 
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performs  as  a bridge  to  serve  the  teaching  and  learning  activities  occurring  between  teachers  and  students.  Based  on 
the  notion,  a regular  Web-based  distance  learning  is  systematized  (See  Figure  1 ). 


Teacher 

• Design  and  post 
course  materials 

Students 

• Regularly  log  in  the 
Web  and  perform 
course  requirements 

WWW 

• Facilitate 
students’  learning 
activities 

• Evaluate  students1 

..  - ..r~ - , _ 

• Participate 
instructional  activities 

• Publish  their  course 
products  on  the  Web 

Figure  1 : A Regular  Web~based  Distance  Education  System 


A Cybercourse  Model 

Based  on  the  above-mentioned  distance  instructional  system,  a Web-based  cybcrcoursc  model  is  designed 
and  formed  by  infusion  four  beneficial  distance  educational  telecomputing  activities  into  such  system.  The  four 
beneficial  distance  learning  acti\  i tics  are  collaborative  learning,  demonstration,  interactive  discussion,  and  problem 
solving.  Following  arc  specific  discussions  on  how  to  apply  these  four  major  beneficial  learning  activities  in  the 
Web-based  distance  learning  system. 

Collaborative  Learning 

Literature  review  suggests  that  collaborative  learning  is  a type  of  instruction  that  could  meet  the  needs  of 
students  of  any  styles  (Ellsworth,  1994).  Collaborative  learning  is  supported  by  the  use  of  research-based  projects 
via  the  Web.  Instructors  could  assign  students  into  either  large  or  small  groups  to  conduct  electronic  researching  on 
course  related  topics. 

Demonstration 

Instructors  could  apply  WWW  as  a demonstration  tool  for  instruction.  Instructors  post  course  syllabus, 
curriculum  materials,  or  students'  products  for  learners  to  read,  discuss,  and  evaluate.  In  addition,  instructors  could 
provide  guidance  about  the  content-related  educational  resources  and  allow7  students  to  conduct  an  electronic  field 
trip  in  viewing  those  resources. 

Interactive  On-line  Discussion 

Interactive  Web-based  discussion  provides  on-going  information  sharing,  facilitates  active  learning,  and 
can  be  controlled  by  the  teachers.  The  Web  with  its  great  capability  offers  various  methods  for  encouraging  on-line 
conversations  between  teachers  and  students.  The  electronic  conversation  between  teachers  and  students  could  he 
made  through  Email,  Listscrv,  newsgroups,  Web  conference  page,  and  synchronous  discussion  (chatting).  Email  is 
the  most  simple,  direct,  and  effective  instructional  strategy  in  a Web-based  distance  cybercourse.  It  can  become  the 
principal  tool  for  maintaining  interactions  hetween  students  and  teachers. 

List>erv  arc  automatic  mailing  lists.  In  the  cybcrcoursc  model,  instructors  could  create  a listscrv  for  their 
specific  class  discussion.  Through  the  listscrv,  instructional  materials  and  messages  could  he  mailed  mandatory  and 
automatically  to  all  of  the  students’  email  addresses  subscribing  to  the  listscrv. 

In  addition,  interactive  electronic  conversation  could  be  implemented  by  creating  a private  discussion 
newsgrojp  to  allow  students  conduct  on-line  discussion  and  post  their  opinions  and  responses.  Newsgroups  could 
also  be  used  to  broadcast  and  post  information  to  students  about  courses,  schedules,  events,  speakers,  and  so  on. 
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Web  conference  pages  offer  another  alternative  for  interactive  on-line  discussion.  Web  conference 
indicates  creating  a Web  site  that  allows  learners  to  submit  their  discussion  contents  and  opinions  in  a designated 
Web  page  so  that  each  student  in  the  class  is  able  to  read  their  peer's  comments  and  ideas  and  respond  his/her  own 
discussion  contents  simultaneously  or  asynchronously. 

Different  from  Email,  synchronous  discussion  (chatting)  allows  both  teachers  and  students  connect  and 
discuss  at  the  same  time.  Such  discussion  provides  a great  group  opportunity  for  distance  learning  via  the  Web. 

Problem  Solving 

Problem  solving  activities  arc  one  of  the  most  beneficial  educational  activities  for  students  of  any  age 
(Ellsworth  1994).  Such  activities  could  be  integrated  hy  Usenet  and  extensive  use  of  the  search  tools,  online 
gathering  from  rich  educational  Web  sites,  and  electronic  information  exchange  tools. 

Accordingly,  with  interweaving  the  four  distance  learning  activities  into  the  regular  Web- based  distance 
education  system,  a cyhcreoursc  model  is  created.  Figure  Two  shows  (he  innovative  model  of  Wch-hased 
cybcrcoursc. 


Teacher 

Students 

• Design  and  post  course 

• Regularly  log  in  the  Web  and 

materials 
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performance 

the  Web 

i 

Figure  2:  The  Cybcrcoursc  Model 


Comparison 

The  significant  difference  between  the  regular  Web-based  distance  learning  system  and  the  innovative 
cybcrcoursc  model  is  the  four  instructional  telecomputing  activities.  The  four  activities  not  only  activate  the  entire 
cyber  learning  but  also  help  instructors  contemplate  about  their  teaching.  Therefore,  two  contrast  perspectives  are 
pointed  out  and  discussed  in  the  following: 

Active  Learning  Versus  Passive  Learning 

The  Web  could  be  a very  passive  tool  if  students  just  browse  the  posted  materials  and  randomly  clicking 
without  any  thoughtful  or  interactive  activities.  If  an  instructor  skillfully  weaves  the  four  telecomputing  activities 
into  his/her  Web-based  distance  instruction,  students'  learning  will  be  totally  different.  Collaborative  learning 
allows  students  to  have  interaction  with  their  classmates  and  engaged  in  the  learning  process.  Electronic 
conversation  provides  excellent  opportunities  to  stimulate  ompuicr-mediaicd  communication  between  teachers  and 
students.  Problem  solving  activity  facilitates  students'  synlbcr  zing  and  org;  i/.ing  skills. 

Mindful  Teaching  Versus  Thoughtless  Teaching 

It  is  always  easy  for  an  instructor  to  follow  and  perform  the  instructional  agenda  listed  in  the  regular  Web- 
hased  distance  learning  system.  However,  to  integral  (he  four  instructional  telecomputing  activities  requires 
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extensive  and  deep  thinking.  Several  questions  have  to  be  considered.  For  example,  how  is  the  collaborative 
learning  going  to  occur  helween  the  remote  students  via  the  Web?  What  types  of  problem-solving  activities  arc  able 
to  meet  the  curriculum  purposes? 

Discussions 

Good  application  of  the  Web-based  cybcrcourse  model  could  promote  not  only  active  teaching  but  also 
effective  learning.  To  ensure  a practical  utilization  of  the  model  in  distance  education,  instructors  should  realize  its 
advantages,  limits,  and  implications. 

Advantages 

Web-based  distance  learning  provides  three  advantages  over  other  distance  educational  technology 
systems.  The  three  benefits  arc:  no  time  and  space  limit,  synchronous  and  asynchronous  communication,  and  linear 
and  nonlinear  learning.  Specific  discussions  about  the  three  advantages  arc  as  follows: 

No  Time  and  Space  Limit 

Through  the  cybercourse,  learning  can  be  taken  place  at  any  convenient  time  and  anywhere  in  the  world. 
Teachers  can  contact  with  any  one  of  their  students  or  students  can  contact  any  one  of  other  students  taking  the  same 
course  early  in  the  day,  after  midnight,  while  traveling,  at  conference,  at  home  or  at  the  office,  without  being  lied  to 
fixed  schedules. 

Synchronous  and  Asynchronous  Communication 

Communication  between  teachers  and  students  could  occur  at  the  same  time  or  at  different  time  in  such  a 
Web-based  distance  course.  The  great  power  of  the  Web-based  instruction  is  that  both  instructors  and  students 
could  express  their  thoughts  and  opinions  at  any  time  and  at  any  location. 

Linear  and  Nonlinear  Learning 

Learning  via  the  Web  allows  students  to  proceed  self-paced  learning.  Students  can  have  great  latitude  in  directing 
their  own  learning  in  a manner  compatible  with  their  learning  styles. 

Disadvantages 

Because  of  the  Web's  advantages  of  allowing  access  at  any  time  and  any  place,  there  are  two  corresponding 
disadvantages.  They  arc  the  difficulty  of  document  students'  performance  and  difficulty  in  tracking  students’ 
exploration  on  the  Web.  Following  is  the  specific  discussions. 

Difficult  of  Documenting  Students’  Performance 

So  far,  it  is  still  very  difficulty  to  document  students'  efforts  or  performance  on  the  Web.  Although  most  of 
the  hypermedia  programs  such  as  HyperCard  arc  able  to  get  an  indication  of  lime  on  task  for  students  using  the 
instructional  software.  There  is  no  easy  way  to  record  students'  time  on  task  while  they  use  the  WWW. 

Difficult  in  Tracking  Students’  Access 

it  is  also  difficult  to  track  whether  students  have  been  to  a Web  site  that  could  mark  as  an  effective  choice 
for  great  learning.  "The  Web's  greatest  inti  .sic  power  is  that  it  encourages  branched  and  nonlinear  instruction 
(Brooks,  1997.  P.  28)."  Students  can  not  only  jump  around  the  materials  that  the  instructors  have  created  for  them, 
hut  also  can  access  information  created  by  ether. 

Considerations 

When  applying  the  Web  as  a distance  learning  medium,  instructors  have  to  take  into  considered  the 
following  three  points.  First,  arc  electronic  conversations  less  effective  than  facc-to-face  conversation?  While 
electronic  conversations  arc  the  major  channel  to  communicate  between  teachers  and  students,  its  instructional  effect 
become  an  essential  issue.  The  importance  of  face-to-face  instruction  has  been  emphasized  in  the  distance  learning 
fields  (Brooks,  1997).  Thus,  an  instructor  might  consider  offering  one  or  two  sessions  for  meeting  with  their 
students  at  remote  sites. 

Second,  docs  Web-based  distance  learning  provide  less  favorable  learning  for  a field  dependent  person? 
Research  studies  indicated  that  students’  learning  style  did  make  a difference  in  their  learning  (Wilkin  et  al,  1967; 


Grecnbowe,  1996).  While  Web-based  distance  learning  is  mainly  delivered  via  electronic  conversation,  will  field 
dependent  learners  perform  poorer  than  the  field  independent  learners  just  because  such  course  offer  less  scenic 
interaction  between  teachers  and  students  than  the  traditional  teaching?  Witkin  (1949)  also  found  that  there  was  a 
significant  sex  difference  in  field-dependent/independent  performances.  Females  tend  to  be  more  field-dependent. 
Accordingly,  an  interesting  research  question  was  generated:  Will  male  students  perform  better  than  female  students 
in  a Web-based  distance  learning  course? 

A final  consideration  regards  to  the  issue  of  “Web-based  misinformation  (Brooks,  1997.  P.  28).“  We  all 
know  that  the  WWW  contains  huge  information  and  resources.  The  quality  of  the  information  resided  in  the  Web 
varies  a lot.  Some  are  with  professional  knowledge;  some  arc  not.  Students  are  quite  possible  to  access 
misinformation  on  the  Internet.  To  avoid  our  students  wasting  too  much  time  in  assimilating  the  misinformation  has 
been  an  enormous  challenge  for  distance  learning  educators. 

Implications  and  Conclusions 

The  Web-based  cybcrcoursc  model  implies  not  only  the  applicability  of  distance  learning  but  also  the 
expansion  in  infusing  technology  into  the  entire  educational  setting.  Undoubtedly,  Web-based  distance  learning 
could  be  applied  to  all  levels  of  education  because  its  rich  resources  containing  knowledge  in  all  levels.  Further 
research  studies  regarding  the  instructional  effects  of  Web-based  distance  learning  will  be  and  should  be 
investigated.  No  matter  what  research  resuits  will  be  generated,  the  Web  will  definitely  continue  to  encourage 
educators  to  integrate  it  in  their  teaching  and  learning  process. 

In  sum,  there  is  no  question  that  Web-bascd  distance  course  with  its  unique  advantages  and  beneficiary 
learning  activities  will  aitract  more  and  more  educators  to  adopt  such  instructional  strategy.  Indeed,  the  Web  with 
its  flexible  and  universal  capabilities  is  just  the  right  tool  to  design  and  develop  an  effective  and  versatile  distance 
educational  course. 
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Abstract 

The  purpose  of  the  study  seeks  ways  of  developing  and  utilizing  more  efficient  Computer-Based 
Instructional  Simulations  (CBIS)  based  on  the  relationship  among  fidelity  levels  (“how  closely  a simulation  imitates 
reality"),  transfer  of  learning,  and  learning  time.  The  hypotheses  presented  in  this  study  suggest  how  to  determine 
the  optimal  shifting  points  of  the  fidelity  levels;  this  method  yields  the  optimal  routes  that  would  help  learners  reach 
the  goals  with  less  time  in  CBIS.  To  verify'  the  hypotheses,  148  seventh  grade  male  students  were  selected 
voluntarily  from  one  junior  high  school  in  Korea,  and  a 4 X 6 (Fidelity  Levels  X Number  of  Lessons)  posttest-only 
experimental  design  was  employed.  Even  though  the  result  failed  to  support  main  hypotheses  due  to  no  distinct 
difference  among  fidelity  levels  of  CBIS,  the  result  shows  that  a cost-effective  point  on  learning  time  exists  in  a given 
level  of  fidelity  of  CBIS.  Therefore,  the  hypotheses  need  to  he  tested  again  in  further  well-designed  experiments  with 
a larger  range  of  fidelity'. 

Theoretical  Background 

Fidelity  means  “how  closely  a simulation  imitates  reality”  (Alcssi  & Trollip,  1991).  The  relationship 
between  the  levels  of  fidelity  and  transfer  of  learning  has  been  an  important  issue  in  the  field  of  training  simulators 
because  the  levels  of  fidelity  arc  deeply  related  with  the  cost  of  the  simulator  (Miller,  1974;  Buam  et  al.,  1982;  Su, 
1984).  In  addition,  controlling  the  levels  of  fidelity  has  been  considered  to  achieve  cost  effective  transfer  of  learning 
(Hays  & Singer,  1989).  Though  the  flexibility  of  switching  fidelity  levels  in  Computer-Based  Instructional 
Simulation  (CBIS)  enhances  the  importance  of  controlling  fidelity  levels  and  its  benefits,  there  arc  few  case  studies. 
Thus,  this  study  focuses  on  how  to  determine  the  optimal  shifting  points  of  the  fidelity  levels  on  the  basis  of  the 
relationship  between  the  amount  of  the  learning  time  and  transfer  of  learning  in  Computer-Based  Instructional 
Simulation  (CBIS). 

Case  studies  on  the  relationship  between  fidelity  and  transfer  of  learning  have  yielded  different  results. 
One  viewpoint  is  that  the  higher  fidelity  of  simulation,  the  better  effect  of  learning  (Ornstcin,  et  al. , 1954; 
Dougherty,  et  al.,  1957,  Allen,  Hays,  & Buffardi,  1986).  These  study  results  are  based  on  the  traditional  theory  of 
identical  elements  (Thorndike,  1913-1914;  Yum,  1931;  Gibson,  1939;  Osgood,  1949).  On  the  other  hand,  a second 
point  of  view  maintains  that  better  transfer  of  learning  may  be  attained  through  lower  levels  of  a simulation  fidelity 
for  students  in  their  early  stages  (Valvcrdc,  1973;  Borcham,  1985),  because  the  higher  fidelity  of  simulation  may 
raise  the  complexity  of  instructions,  which  decreases  the  effect  of  learning  (Alcssi,  1988). 

The  synthesized  assumption  was  presented  by  Miller  (1974),  Alessi  (1988),  and  Hays  and  Singer  (1989). 
They  suggested  that  the  levels  of  fidelity  are  decided  according  to  the  “stages  of  learning,1’  that  is,  “the  student’s 
current  instructional  levels,”  and  the  most  cost  effective  simulation  learning  is  that  the  higher  levels  of  fidelity 
replace  thelower  levels  as  the  instructional  levels  of  students  improve.  In  addition,  Roscoe  (1971)  and  Povenmirc 
and  Roscoe  (1973)  have  shown  that  the  cost  effectiveness  of  learning  in  the  certain  level  of  fidelity  is  inversely 
proportional  to  the  amount  of  learning  time. 

Research  Questions 

This  study  supposes  that  each  level  of  fidelity  would  draw  a different  increasing  curve  of  transfer  of 
learning  according  to  the  learning  time  (Hypothesis  I)  as  presented  Figure  1.  If  the  assumption  of  Figure  1 is  true, 
the  optimal  shifting  time  from  Fn-1  to  Fn,  or  from  Fn  to  Fn+1 , would  be  determined  at  the  point  where  the  gap  of 
transfer  of  learning  between  the  consecutive  two  levels  of  fidelity  (Fn-1  and  Fn,  or  Fn  and  Fn+1)  is  maximum 
(Hypothesis  II,  Figure  2).  Therefore,  the  suitable  shifts  of  the  fidelity  levels  according  to  the  learning  time  would 
have  learners  reach  the  goal  with  less  time  in  CBIS. 
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Figure  2.  Optimal  shifting  of  fidelity  levels  and  its  benefit  (Hypothesis  II) 

Note:  Fn:  Given  level  of  fidelity.  Fn- 1 : Lower  fidelity  than  Fn.  Fn+ 1 : Higher  fidelity  than  Fn 
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Research  Method 

Sampling  and  Experimental  Design 

To  verify  the  above  hypothesis  1 presented  in  Figure  1,  148  seventh  grade  male  students  were  selected 
voluntarily  from  one  junior  high  school  in  Korea.  They  had  already  taken  the  same  classes  about  the  basic  use  of 
the  Windows  3.1  system  in  IBM  compatible  computers.  For  the  experiment,  a 4 X 6 (Fidelity  Levels  X Number  of 
Lessons)  posttest-only  design  was  employed. 

Operational  Definitions 

In  this  experiment,  fidelity  level  means  the  number  and  flexibility  of  actions  that  the  student  may  take  in 
operating  simulation.  The  learning  time  is  the  number  of  simulation  lessons  that  the  student  took.  Transfer  of 
learning  refers  to  the  amount  of  profit  generated  by  the  student  through  the  manipulation  and  utilization  of  several 
variables  during  the  posttest  business  simulation. 

Material 

The  content  of  instruction  was  limited  to  basic  knowledge  and  common  sense  in  economics,  which  is 
treated  as  a social  science  subject  in  the  fifth  and  eighth  grades.  The  students  ran  the  simple  fruit  store  in  CBIS.  A 
CB1S  program  with  four  different  levels  of  fidelity  was  developed.  Table  I shows  the  standards  used  for  classifying 
the  different  levels  of  fidelity  in  the  CBIS  development  process. 


Table  1.  The  standards  for  classifying  the  levels  of  fidelity 


The  factors  of  classifying 

The  levels  of  fidelity 

m 

2 

3 

4 

E* 

Number  of  action 

Determination  of  fruit  order  quantities 

I8JMI 

Yes 

Yes 

Yes 

Determination  of  fruit  sale  prices 

No 

Yes 

Yes 

Yes 

Yes 

Selection  of  fruit  types 

No 

No 

Yes 

Yes 

Yes 

Flexibility  of  action 

Number  of  fruit  types  to  saic/ 

Total  number  of  fruit  types  to  select 

NA 

NA 

1/2 

2/3 

2/3 

Note:  *E  means  the 

fidelity  level  of  posttest  simulation. 

Procedure 

The  students  were  randomly  assigned  to  the  four  groups  of  differing  levels  of  fidelity,  and  they  practiced  the 
CBIS  set  at  their  group’s  level  of  fidelity  for  two  minutes.  Immediately  thereafter,  they  were  evaluated  for  two 
minutes  by  the  posttest  simulation,  which  was  the  highest  level  of  fidelity.  In  this  manner,  two  minutes  practice  and 
two  minutes  test  were  repeated  six  limes  to  estimate  the  curve  of  transfer  of  learning.  The  responses  of  the  students, 
the  net  profits  during  the  test  simulation,  were  recorded  in  the  computer. 

Results 

(IJ  As  presented  Figure  3,  there  were  no  differences  among  the  four  increasing  curves  of  transfer  of 
learning  which  each  level  of  fidelity  drew  through  the  learning  time.  [2]  Meanwhile,  the  transfer  of  learning  for  the 
CBIS  varied  with  the  six  gradual  stages  of  learning  time,  F (5,  739)=36.3.  pc.001.  The  post-hoc  analyses  indicated 
that  the  transfer  of  learning  curve  marked  a sharp  rise  at  the  first  and  second  simulation  lessons  (learning  lime)  with 
statistical  significance,  and  then  it  flattened  out  after  the  second  lesson  (Figure  3). 
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Figure  3.  Relationship  between  transfer  of  learning  and  learning  time  according  to  fidelity  levels  (Result) 

Discussion  & Implication 

The  result  [2]  supported  by  Roscoc(197!)  and  Povcnmirc  and  Roscoc(I973)  implies  that  CBIS  has  the 
cost-effective  point  on  the  amount  of  learning  time.  In  this  ease,  it  is  most  cost-effective  to  stop  instruction  after  the 
second  simulation  lesson. 

The  result  [1]  has  two  possible  interpretations.  First,  the  lack  of  an  effect  by  the  levels  of  fidelity  on  the 
transfer  of  learning  in  this  experiment  may  be  attributable  to  the  lack  of  any  real  differences  in  the  levels  of  fidelity 
of  the  CBIS  developed.  Actually,  this  assumption  (Figure  1)  is  based  on  Alessi’s  comprehensive  model  (1988);  the 
fidelity  levels  should  be  determined  by  a variety  of  aspccts-undcrlying  models,  presentations,  user  actions,  and 
system  feedback-  and  a wide  range  of  situations-form  the  low  level  fidelity  to  the  actual  experience.  In  this 
experiment,  however,  the  four  different  levels  of  fidelity  considered  only  the  user  action  aspect,  and  the  close 
intervals  of  fidelity  levels  were  all  admittedly  in  a low  range  due  to  limited  financial  resources.  Therefore,  the 
assumptions  (Figure  1 and  2)  will  be  tested  again  in  further  well-designed  researches  with  a larger  range  of  fidelity. 

The  second  interpretation  is  that  the  levels  of  fidelity  may  be  not  important  variables  on  transfer  effects  in 
CBIS.  The  researches  and  claims  of  Cox,  et  al . (1965),  Grimslcy  (1969),  Hopkins  (1975),  and  Johnson  (1981)  in  the 
field  of  simulators  may  be  regarded  as  being  consistent  with  the  results  of  this  study.  In  addition,  psychological 
aspects  of  fidelity  may  be  more  important  than  the  physical  aspects  of  it  (Prather,  1973;  Dittrich,  1977).  Finally, 
psychological  aspects  of  fidelity  in  CBIS  will  have  to  be  considered  in  further  researches. 
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Abstract 

The  purpose  of  this  study  was  to  examine  the  differential  effects  of  computer  conferencing  on  expository 
writing  for  students  (n  = 109)  of  seven  intelligence  types . Students  were  assigned  to  four  treatment  groups  that 
provided  controlled  exposure  to  a writing-prompt:  unstructured  exposure , computer  conferencing , face-to-face 
discussion,  and  computer  conferencing  combined  with  face-to-face  discussion.  Following  treatment  all  students 
wrote  an  essay  on  the  writing-prompt.  MANOVA  indicated  that  participation  in  computer  conferences  did  not 
significantly  improve  scores  on  essays.  However  some  interactions  between  treatments  and  intelligence  type  were 
significant.  Results  indicate  that  intelligence  type  is  an  important  instructional  variable  when  implementing 
computer  conferences. 

Background 

The  application  of  computer  conferencing  as  a teaching/lcarning  tool  is  increasing  dramatically  as  more 
educational  institutions  gain  access  to  the  Internet.  Several  researchers  have  found  that  computer  conferencing  (CC) 
can  be  a powerful  force  for  facilitating  discussion  and  for  encouraging  writing  (Kieslcr,  Siegel,  &.  McGuire,  1984; 
Rafaeli  & Sudwccks,  1996;  Tagg  & Dickinson,  1995).  Research  on  computer  conferencing  has  been  based  largely 
on  analysis  of  conference  transcripts  and  results  of  attitude  surveys.  For  instance,  in  one  study  discourse  analysis 
revealed  that  CC  contributed  to  preservice  teaching  apprenticeship  and  learning  of  educational  psychology  by 
providing  for  positive  communication  during  early  Held  experiences:  ‘‘students  were  heavily  involved  in  electronic 
learning. ...and  teachers  electronically  scaffolded  or  apprenticed  learning  ...without  giving  away  answers”  (Bonk, 
Malikowski,  Angcli.  & East,  1997).  There  has  been,  however,  little  investigation  of  external  outcomes  of  computer 
conferencing.  In  this  study  we  investigated  the  effects  of  CC  on  expository  writing  beyond  the  CC. 

The  researchers’  assumptions  about  the  postitivc  effects  of  CC  on  learning  as  reflected  in  expository 
writing  were  based  upon  constructivist  learning  theory  which  indicates  that  providing  learners  with  opportunities  for 
written  reflection,  interaction,  shared  perspectives,  and  mentorship  each  contribute  positively  to  knowledge 
construction  (Driscoll,  1994).  In  CC  participants  grapple  with  their  ideas  and  think  through  their  entries  before 
carefully  constructing  messages  and  replies  to  others’  messages.  They  read  each  others’  ideas  and  responses  to  their 
entries.  On-line  dialog  that  results  provides  students  with  the  opportunity  to  test  and  refine  their  understandings  in  an 
ongoing  process. 

Multiple  Intelligences 

In  addition  to  investigating  the  comparative  effects  of  face-lo-facc  (f-f)  discussion,  CC,  and  a combination 
of  CC  and  f-f  discussion,  wc  included  a second  facet  to  our  study:  investigation  of  the  interactive  effects  of  CC  and 
multiple  intelligences  on  student  learning.  In  a similar  study,  Brenner  and  Hill  (1997)  explored  the  comparative 
effects  of  asynchronous  distance  learning  on  achcivement  for  field  dependent  versus  field  independent  learners  and 
found  no  interaction.  However,  the  theory  of  multiple  intelligences  indicates  that  students’  individual  intelligence 
types  must  be  addressed  through  varied  strategics  (Gardner,  1993).  Therefore,  multiple  intelligences  have  become 
important  variables  to  instructional  designers  and  have  implications  for  the  use  of  CC  for  learning.  The  seven 
intelligences  identified  by  Gardner  arc--  linguistic,  logical/mathematical,  spatial,  bodily/kincsthctic,  musical, 
interpersonal,  and  intrapcrsonal.  Interactions  between  type  of  intelligence  and  instructional  media  applications  need 
to  be  identified  so  that  alternative  leaching  strategies  can  be  applied  lo  learners  according  to  their  lypc(s)  of 
intelligence.  Because  CC  provides  for  linguistic,  interpersonal  learning  vve  expected  learners  of  different 
intelligences  to  be  differentially  effected  by  CC. 
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Research  Questions 

In  th a study  researchers  investigated  the  effects  of  CC  on  student  expository  writing.  We  asked  whether  or 
not  students  learned  from  their  on-line  discussions,  above  and  beyond  f-f  discussions.  Specifically-  1)  did  students 
who  participated  in  a computer  conference  on  a particular  topic  write  better  essays  on  that  topic  than  students  who 
did  not  participate  in  a computer  conference  on  that  topic,  and  2)  did  students  who  participated  in  the  combination  of 
a f-f  discussion  and  a computer  conference  on  a particular  topic  write  better  essays  on  that  topic  than  students  who 
did  not  participate  in  a computer  conference  and  f-f  discussion  on  that  topic?  In  addition  we  asked,  3)  did  computer 
conferences  differentially  affect  writing  for  students  of  different  intelligence  types.  We  also  attempted  to  identify 
process  variables  that  contributed  to  effective  on-line  and  f-f  discussions. 

Based  upon  constructivist  theory  we  expected  CCs  and  f-f  discussion  to  positively  affect  writing  above  and 
beyond  the  effects  of  unstructured  study  or  f-f  discussion  alone.  In  addition, multiple  intelligence  theory  connoted 
that  unstructured  study  time,  CC,  and  f-f  discussion  would  have  differential  effects  on  learning  as  reflected  in 
expository  writing.  Specifically,  we  expected  that  students  with  linguistic,  logieo-mathcmatical,  spatial,  and 
interpersonal  intelligence  types  would  be  positively  effected  by  CC  while  students  with  musical,  bodily-kinesthetic, 
and  intrapcrsonal  intelligence  types  would  be  unaffected  or  negatively  affected  by  CC.  We  made  these  hypotheses 
because  CC  is  a print-based,  linguistic,  writing  activity  that  involves  collaborative  construction  of  meaning. 

Methods 

Ninety-nine  undergraduate  students  enrolled  in  an  introductory  educational  technology  course  in  spring 
1997  at  Texas  A&M  University  were  divided  into  four.  11  day  CCs  in  which  they  discussed  different  topics  on 
educational  technology.  They  used  FirstClass™  software  to  conduct  CCs  which  were  moderated  by  the  instructor 
who  also  led  classes  in  their  f-f  discussions  of  the  topics. 

The  moderator’s  role  in  both  CCs  and  F-F  discussions  was  to  provide  structure,  keep  students  on  topic,  and 
weave  threads  of  the  discussion  (Berge,  1995;  Eastmond,  1992).Thc  CC  structure  was  such  that  the  instructor 
entered  the  questions  in  FirstClass  for  Ihe  writing-prompt  for  the  CC  group  and  CC/F-F  groups.  With  the  exception 
of  brief  comments  from  the  moderator  and  weaving  in  each  of  the  groups,  students  had  free  reign  to  discuss  what 
ever  topics  emerged.  The  moderator  attempted  to  control  for  extended  tangential  discussions  by  creating  new  folders 
for  those  discussions. 

The  CCs  in  this  study  involved  reading  a discussion  topic  that  was  entered  by  the  instructor,  entering  a 
response  to  that  topic  in  the  CC,  reading  other’s  responses,  and  replying  to  those  responses.  In  contrast  to 
experiences  in  facc-to-facc  (f-f),  participants  had  opportunities  to  reflect  upon  their  ideas.  As  a result,  responses 
were  likely  to  be  less  spontaneous  and  more  deliberate  than  in  f-f  discussion.  They  were  also  likely  to  demonstrate 
deeper  understanding  and  be  more  clearly  staled. 

Study  Design 

In  this  study  we  used  quantitative  and  qualitative  methods  to  answer  the  research  questions.  A "Posttest- 
Only  Control  Group  Design"  with  four  treatment  groups  was  used  to  explore  the  comparative  effectiveness  of 
unstructured  study  time  (control  group),  f-f  discussion  only,  CC  discussion  only,  and  both  f-f  and  CC  discussion. 
Participants  were  not  randomly  assigned  to  groups.  Instead  they  received  treatment  within  their  lab  sections.  The 
writing-prompt  discussion  topic  was-  What  impact  might  the  combination  of  instructional  design,  media,  and 
computing  have  on  learning?  A placebo  discussion  topic  was  used  in  CCs  and  f-f  discussions  for  treatments  that  did 
not  include  CC  or  f-f  discussion  of  the  writing-prompt.  The  placebo  writing-prompt  was--  What  classroom 
management  .strategics  might  you  use  for  integrating  technology  in  the  curriculum? 

Students  were  assigned  to  four  treatment  groups: 

1 ) Control  Received  no  exposure  to  the  writing-prompt  discussion  topic  via  CC  or  f-f  discussion.  However,  they  did 
receive  exposure  to  the  writing-prompt  discussion  topic  through  readings,  lecture,  assignments,  and  unstructured 
study  time. 

2)  F-F  Participated  in  a 20  minute  f-f  discussion  on  the  writing-prompt  discussion  topic  and  received  exposure  to  the 
writing-prompt  discussion  topic  through  readings,  lecture,  and  assignments. 

3)  CC  Participated  in  a discussion  of  the  writing-prompt  in  a moderated  1 1 day  computer  conference  and  received 
exposure  to  the  writing-prompt  discussion  topic  through  readings,  lecture,  and  assignments. 

4)  F-F/CC  Participated  in  a discussion  of  the  writing-prompt  topic  in  a moderated  1 1 day  computer  conference, 
participated  in  a 20  minute  class  discussion  on  the  writing-prompt  discussion  topic,  and  received  exposure  to  the 
discussion  topic  through  readings,  lecture,  and  assignments. 
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Data  Sources 

The  data  sources  included  contents  of  CCs,  the  essay  scores,  and  multiple  intelligences  inventories. 
Average  number  of  entries  and  sittings  to  make  entries  were  determined  for  each  treatment  group.  Scores  on  essays 
written  about  the  writing-prompt  following  treatment  were  determined  using  the  instrument,  The  Composition 
Profile  (Hughey,  J.B.  & Wormuth,  D.R.  1985,  r=.98).  The  dependent  variables  were  scores  on  the  quality  of  the 
contents,  organization,  and  overall  composition  of  essays  on  the  writing-prompt  discussion  topic.  Students'  essays 
were  scored  independently  by  three  readers  trained  in  the  use  of  the  instrument.  Face,  content,  and  construct  validity 
have  been  established  for  the  instrument. 

We  established  rater  reliability  for  the  essays  by  having  two  readers  assess  each  paper  according  to  content, 
organization,  and  composition  score  (a  composite  of  content  and  organization).  We  assessed  the  reliability  using 
Pearson  Product  Moment  formula.  Reader  reliability  was  .92  for  content,  .78  for  organization,  and  .91  for 
composition  with  a 1%  discrcpcney  rate. 

The  self-report  Multiple  Intelligences  Inventory  (Mil)  was  administered  and  participants’  types  of 
intelligences  were  identified  through  scoring  this  instrument  (Armstrong,  1993).  Students  were  classified  as  having 
one  of  the  seven  intelligences  if  they  selected  six  or  more  of  the  ten  possible  behaviors  used  to  describe  each 
intelligence. 

Results 

The  CCs  contained  344  messages  from  participants  and  24  moderator  messages.  Participants  and 
moderators  differed  in  their  degree  of  participation  in  the  CC  across  groups.  The  group  that  computer  conferenced 
without  a f-f  discussion  on  the  writing  prompt  participated  slightly  more  than  the  other  groups.  The  moderator 
appears  to  have  been  most  active  in  the  CC  & F-F  group  making  nine  entries  in  eight  sittings  (see  Table  1 ). 

Table  L Activity  in  Computer  Conferences  by  Groups 


Participants 

N 

# of  entries/ 
avc.  # per  person 

# of  sittings/ 
ave.  per 
person 

Mod. 

entries 

Mod. 

sittings 

Nonparlicipalio 
n in  CC 

Control 

17 

66/4 

54/3 

3 

3 

1 

F-F 

28 

75/3 

63/2 

6 

6 

1 

CC 

28 

115/4 

93/3 

6 

6 

0 

CC  & F-F 

26 

88/3 

65/4 

9 

8 

2 

Although  participants  were  not  assigned  to  groups  according  to  their  types  of  intelligence,  the  types 
dispersed  across  groups  fairly  evenly.  However,  the  control  group  is  the  only  group  with  strength  in  linguistic 
intelligence  and  they  are  stronger  overall  in  the  intelligences.  Table  2 shows  that  the  control  group  had  linguistic, 
spatial,  bodily-kincslhitic,  musical,  and  interpersonal  intelligences.  The  other  three  treatment  groups  had  bodily- 
kincsthitic,  musical,  and  interpersonal  intelligences. 


Table  2 . Mean  Scores  on  Multiple  Intelligence  Inventory  by  Groups 


MI  Control 


Mean 

SD 

L 

5.8 

2.2 

LM 

3.9 

2.6 

S 

5.5 

2.5 

BK 

5.6 

2.9 

M 

6.7 

2.9 

TER 

6.1 

2.6 

TRA 

4.9 

2.2 

Total 

38.5 

F-F  CC 


Mean 

SD 

Mean 

4.1 

1.9 

4.5 

4.4 

2.8 

4.1 

4.6 

2.2 

5.4 

5.8 

1.9 

5.9 

5.5 

2.8 

7.0 

5.6 

2.4 

6.1 

3.9 

1.5 

4.5 

33.9 

37.5 

CC&F-F 


SD 

Mean 

SD 

2.4 

4.7 

2.2 

3.2 

4.2 

2.0 

2.1 

4.5 

1.8 

2.2 

5.7 

2.4 

2.4 

6.0 

2.4 

2.3 

5.5 

2.4 

2.0 

4.6 

2.0 

35,2 

A MANOVA  was  used  to  determine  if  differences  between  treatment  groups  existed  and  if  the  treatments 
interacted  with  intelligence  types.  Follow-up  correlations  indicated  how  treatments  interacted  with  intelligence 
types.  The  independent  variables  of  teatmenl  group  and  intelligence  type  were  tested  across  the  three  dependent 
variables:  content  of  essay,  organization  of  essay,  and  overall  composition  of  essay.  Treatment  groups  did  not  differ 
significantly  on  expository  writing.  Students  who  discussed  the  writing-prompt  in  their  CC  and/or  f-f  discussion  did 
not  perform  significantly  better  on  the  three  measures  than  those  who  did  not  discuss  the  writing-prompt  in  a CC 
and/or  f-f  discussion  (see  Tabic  3). 

Table  3.  Scores  on  Content , Organization  and  Composition  by  Groups 


GR 

N 

Mean 

SD 

Mean 

SD 

Mean 

SD 

Content 

Content 

Organ  i /.alio 

Organizatio 

Comp. 

Comp. 

n 

n 

Control 

17 

21.32 

4.03 

14.85 

2.78 

36.18 

6.55 

F-F 

28 

20.08 

3.93 

13.67 

2.42 

33.78 

6.1 1 

CC 

28 

21.75 

3.70 

14.93 

1.95 

36.68 

5.15 

CC&F-F 

26 

22.71 

3.92 

15.17 

2.55 

37.88 

6.18 

Computer  conferences  and  f-f  discussions  differentially  affected  learning  for  students  with  different 
intelligence  types.  Students  with  interpersonal  intelligence  who  participated  in  f-f  discussions  but  did  not  computer 
conference  on  the  writing-prompt  benefited  from  such  treatment.  They  out  performed  other  students  in  writing 
content,  organization,  and  overall  composition.  Students  with  intrapcrsonal  intelligence  were  negatively  affected  by 
CC  and  f-f  discussion  on  both  writing  content  and  overall  composition.  However,  CC  alone  and  f-f  alone  did  not 
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negatively  affect  inlrapersonal  intelligence  types’  writing.  Students  with  bodily-kinesthetic  intelligence  were 
negatively  affected  on  overall  composition  by  CC  without  f-f  discussion  (see  Table  4). 

Given  the  control  group’s  linguistic  and  spatial  intelligence  we  would  expect  them  to  perform  well  on  an  essay. 
Differences  between  groups'  intelligences  may  provide  an  explanation  for  the  lack  of  difference  between  the  control 
group  and  the  other  groups,  particularly  the  CC  & f-f  group.  Much  to  our  surprise,  students  with  linguistic 
intelligence  appear  to  be  unaffected  by  the  treatment.  We  can  hypothesize  that  the  control  group  did  well  on  their 
content,  organization  and  overall  composition  because  of  their  strong  multiple  intelligence  and  because  they  were 
less  needy  of  CC  and  f-f  discussion  to  support  their  learning  of  the  writing  prompt  during  unstructured  study  time. 


Table  4.  Multivariate  Analysis  of  Variance  of  Scores  on  Content,  Organization,  and  Overall  Composition  by 
Multiple  Intelkgence 


Source 

Variable 

SS 

df 

MS 

F 

Sig. 

Group(GP) 

Content 

96.00 

3 

32.00 

2.12 

.102 

Organization 

36.35 

3 

12.11 

2.10 

.105 

Composition 

244.54 

3 

81.51 

2.29 

.082 

GP*L 

Content 

26.20 

4 

6.55 

.52 

.716 

Organization 

32.22 

4 

8.05 

1.62 

.177 

Composition 

83.53 

4 

20.88 

.727 

.577 

GP*LM 

Content 

54.55 

4 

13.63 

1 .09 

.365 

Organization 

17.05 

4 

4.26 

.86 

.492 

Composition 

1 18.7 

4 

29.6 

1 .03 

.396 

GP*S 

Content 

88.02 

4 

22.00 

1.77 

.144 

Organization 

15.01 

4 

3.75 

.758 

.556 

Composition 

170.91 

4 

42.73 

1.48 

.216 

GP*BK 

Content 

123.23 

4 

30.80 

2.48 

.052 

Organization 

45.45 

4 

11.36 

2.29 

.068 

Composition 

308.17 

4 

77.04 

2.68 

.039* 

GP*M 

Content 

92.0i 

4 

23.00 

1.85 

.129 

Organization 

34.92 

4 

8.73 

1.76 

.146 

Composition 

223.99 

4 

55.99 

1.94 

.1  12 

GP*TER 

Content 

126.93 

4 

31.73 

2.55 

.046* 

Organization 

61.61 

4 

15.40 

3.10 

.021* 

Composition 

351.82 

4 

87.95 

3.06 

.022* 

GP*TRA 

Content 

144.31 

4 

36.07 

2.90 

.028* 

Organization 

44.35 

4 

11.08 

2.23 

.074 

Compost  it  ion 

333.63 

4 

83.40 

2.90 

.028* 

*p  >.05 

L=Unguistic,  LM=LogicaI/Malhcmatical,  S=Spalial,  BK=Bodily  Kinesthetic,  M=Musical,  TER=Intcrpcrsonal, 
TR  A=I  ri  traperson  al 
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Another  possible  explanation  for  the  lack  of  effects  of  CC  is  that  the  writing  prompt  may  have  been  too  . 
easy  for  participants  to  address  during  unstructured  study  of  the  text  and  lecture  notes.  A follow-up  study  using  a 
writing  prompt  on  a topic  that  no  participants  have  had  exposure  to  might  provide  belter  understanding  of  the  effects 
of  CC  on  writing. 

Educational  Significance  and  Implications 

Evidence  from  this  study  indicates  that  CCs  do  not  affect  expository  writing  about  a specific  writing 
prompt  above  and  beyond  f-f  discussions  or  unstructured  study.  While  the  writing  prompt  did  iv  1 appear  to  be 
learned  through  CC  and/or  f-f  discussion,  except  for  for  interpersonal  learners,  we  still  value  CC  as  ..  icilitator  of 
collaborative  learning.  Students  used  the  medium  to  address  topics  of  interest  to  them  and  collaborativcly 
constructed  meaning  about  topics  of  their  choice.  We  had  a specific  objective  in  mind  for  learning  in  the  CC. 
Students  look  tangents  from  this  objective  as  they  discussed  content  of  special  interest  to  them.  Collaborative,  social 
learning  environments  such  as  CC  may  not  provide  an  atmosphere  for  learning  specific  objectives. 

Therefore,  a follow-up  study  involving  content  analysis  would  be  appropriate  for  identifying  what  was 
learned  in  these  CCs.  In  addition,  findings  indicate  that  computer  conferences  might  negatively  affect  writing  about 
a specific  writing  prompt  for  students  with  different  types  of  intelligence.  Specifically,  CC  may  interfere  with 
bodily-kinesthetic  and  inlrapcrsonal  learners’  abilities  to  write  about  a specified  topic.  Alternative  media  should  be 
considered.  These  findings  call  for  future  investigation  into  specific  facilitative  strategies  that  might  be  employed 
within  CCs  in  order  to  attain  specific  objectives  and  reach  different  types  of  learners. 

This  study  expands  the  research  base  related  to  the  impact  of  CC  on  student  performance.  We  determined 
that  intelligence  types  and  CC  interact  to  create  a differential  effect.  Such  findings  speak  to  instructional  designers 
regarding  strategies  for  individualized  instruction. 
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Abstract 

This  case  study  focuses  on  the  effects  of  collaborative  activities  between  two  teachers  and  their  students. 
We  explored  the  effectiveness  of  distance  learning  for  adolescents  in  promoting:  I)  self-esteem , 2)  achievement , and 
3)  multicultural  understanding.  In  Cultural  Connections,  diverse  students  across  Texas  collaborated  on 
multicultural  activities  which  helped  them  grow  in  self-esteem,  achievement , and  multicultural  understanding.  This 
project  demonstrates  that  in  networked  classrooms  students  can  connect  with  distant  others  to  learn  about  and  from 
their  perspectives  and  to  increase  their  multicultural  understandings.  In  addition , distance  technologies  can  foster 
team  teaching  across  cultures  and  geographical  distances. 

Background  and  Theoretical  Perspective 

In  this  case  study,  we  explored  the  effectiveness  of  distance  learning  for  adolescents  in  promoting:  1)  self- 
esteem, 2)  achievement,  and  3)  multicultural  understanding.  In  our  project,  Cultural  Connections,  predominantly 
Hispanic  students  collaborated  with  diverse  students  in  various  schools  via  interactive  videoconference,  desktop 
videoconference,  and  computer  conference. 

The  theory  of  transformative  pedagogy  suggests  that  the  fundamental  purpose  of  education  is  to  promote 
personal  growth  in  individuals.  As  individuals  grow  through  education,  they  transform  into  responsible,  caring 
citizens  who  affect  society  positively.  To  facilitate  building  a broad  world-view  in  students,  educators  need  to 
provide  collaborative  learning  experiences  for  social  construction  of  meaning.  Telecommunications  can  be  used  to 
expand  the  range  of  exposure  to  multicultural  interactions  (Cummins  & Sayers,  1995).  In  addition,  multimedia 
technologies  can  be  used  for  visual/vcrbal  sharing  of  ideas  across  distances. 

Distance  technologies  expand  the  range  of  opportunities  for  students  to  build  relationships  with  people  of 
different  cultures.  In  the  current  social-constructivist  conception  of  learning,  “education  is  the  shared  way  of 
thinking  about  one’s  self,  the  community,  and  the  world”  (Riel,  1995,  p.  219).  Schools  might  play  a significant  role 
in  nurturing  students’  positive  identity  formation  by  providing  them  with  opportunities  to  build  distant  relationships. 
In  order  to  become  more  tolerant  and  respectful  citizens,  students  need  to  develop  relationships  with  people  from 
diverse  cultures  and  backgrounds  (Moffett,  1994).  Geographical  and  cultural  isolation  can  limit  opportunities  for 
relationship  building  beyond  one’s  culture.  However,  cross-classroom  collaborates  allow  students  to  connect  with 
distant  others  from  around  the  world.  The  collaborative  learning  process  has  the  potential  to  transform  individual 
participant’s  perspectives  from  parochial  to  global  (Cummins  & Sayers,  1995;  Metcalf,  1994). 

Cultural  Connections  is  an  open  forum  between  distant  classes  that  use  telecommunications  and  multimedia 
technologies  for  collaborative  learning.  In  collaboration  with  university  researchers,  24  teachers  from  three  schools 
developed  several  identity  forming  and  multicultural  curricular  activities  for  their  students  to  conduct  over  the 
distance.  For  instance,  they  developed  multimedia  activities  from  writing  activities  in  l Thought  I Was  from  Another 
Planet  (Dresser,  1994),  a book  of  short  stories  and  writing  activities  designed  to  help  students  develop  authentic 
writing  voices  and  heighten  their  sensitivity  to  our  multicultural  world.  Students  did  the  activities,  which  involved 
writing  and  distance  collaboration,  and  ultimately  produced  multimedia  presentations  of  their  work  which  they 
shared  with  their  partners  over  the  distance. 

In  another  activity,  students  participated  in  a distance,  open  forum  to  discuss  the  pros  and  cons  of  school 
uniforms.  Students  conducted  library  and  Internet  research  on  the  topic;  they  surveyed  campus  students  and  faculty; 
they  calculated  means;  and  they  formulated,  developed,  and  presented  persuasive  arguments  across  the  distance. 

In  Cultural  Connections,  multimedia  software  and  distance  technologies  were  used  to  eliminate  isolation. 
Student  development  of  multimedia  portfolios  and  cross-classroom  collaboration  were  meant  to  foster  the  building 
of  multicultural  relationships  while  honoring  the  strength  of  diversity  and  emphasizing  how  similarities  unify. 
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Research  Questions 

Participating  teachers  conducted  at  least  nine  distance  learning  experiences  with  their  students  over  a 
school  year.  In  this  study  we  asked-  In  what  ways,  if  any,  did  the  distance  learning  experiences  affect  1)  self- 
esteem, 2)  school  achievement,  and/or  3)  multicultural  understanding  in  participating  students? 

Methods 

Cultural  Connections  included  Berta  Cabaza  Middle  School  in  San  Benito,  Texas,  which  is  located  in  the 
Lower  Rio  Grande  Valley;  Jones  Intermediate  School  in  Waller,  Texas,  which  is  located  about  350  miles  to  the 
north  of  San  Benito;  and  Somerville  Junior  High  School  in  Central  Texas.  Participants  in  this  study  included  24 
teachers  and  the  24  students  of  2 of  those  teachers,  2 school  counselors,  3 administrators,  and  2 university  faculty  at 
Texas  A&M  University.  Sixteen  teachers  at  Berta  Cabaza,  six  teachers  at  Jones  Intermediate  and  two  teachers  at 
Somerville  Junior  High  who  teach  math,  science,  language  arts  and  social  studies  conducted  Cultural  Connections 
curricular  activities.  A counselor  at  Berta  Cabaza  partnered  with  a counselor  at  Somerville  Junior  High  to  conduct 
guidance  activities.  School  principals  and  university  faculty  facilitated  the  project.  All  student  populations  in 
partcipaling  schools  were  racially  and  culturally  diverse,  and  participating  students  equitably  represented  the 
ethnicity  of  the  student  populations  of  each  school.  The  school  and  university  partners  were  equipped  with 
interactive  compressed  video  systems,  HyperStudio™  software,  Internet  connections,  CU-SecMc™  software,  and 
FirstClass™  computer  conferencing  software. 

This  case  study  focuses  on  the  effects  of  collaborative  activities  between  two  teachers  and  their  students. 
The  two  teachers,  one  in  San  Benito  and  the  other  in  Waller,  partnered  for  planning  and  implementation  of 
curricular  activities  with  their  students.  They  conducted  the  ongoing  activities  between  October  and  June  of  1996 
and  1997.  Fourteen  randomly  placed  eighth  grade  students  in  San  Benito  and  ten  gifted  and  talented  fifth-grade 
students  in  Waller  were  active  in  the  project  throughout  the  1996-1997  academic  year. 

The  investigation  relied  on  ethnographic,  case  study  methods.  To  determine  effects  of  distance  learning  on 
self-esteem,  achievement,  a.id  multicultural  understanding,  wc  conducted  content  analyses  of  each  of  the  data 
sources.  Wc  looked  for  emergent  themes  and  gained  consensus  on  the  extent  to  which  the  data  sources  revealed 
answers  to  the  research  questions.  Data  sources  included  participants’  reflections,  contents  of  videoconferences, 
students’  HyperStudio  portfolios,  interviews 'of  the  students  early  in  the  school  year  and  again  at  the  end  of  the  year, 
and  evaluations  of  the  Annual  Distance  Learning  Videoconference  attended  by  the  24  teachers. 

In  addition,  to  estimate  the  effects  of  Cultural  Connections  on  achievement,  wc  examined  scores  on  three 
school  benchmarking  reading  tests  at  Berta  Cabaza  in  San  Benito.  The  tests  were  administered  to  all  students  in  the 
school  as  part  of  the  school’s  accountability  system.  Only  one  class  participated  in  Cultural  Connections.  Wc 
compared  the  scores  of  14  students  in  the  Cultural  Connections,  distance  learning  class  with  the  scores  of  the 
students  in  the  same  teacher’s  classes  that  did  not  participate  in  Cultural  Connections. 

For  purposes  of  this  research  we  limited  the  definition  of  multicultural  understanding  to  demonstrations  of 
appreciation  of  others  and  breadth  of  world-view.  To  determine  the  degree  of  multicultural  understanding,  wc 
analyzed  contents  of  interviews  and  participating  students’  multimedia  portfolios.  The  portfolios  contained 
reflections  about  students’  homes,  thoughts  and  values,  goals,  likes  and  abilities,  and  stories.  Students  shared  the 
contents  of  their  portfolios  via  videoconference  to  partners  at  distant  schools.  The  portfolios  were  under  continuous 
construction  and  reconstruction  by  the  students  and  served  as  valuable  disclosures  of  participants’  feelings. 

We  each  color-coded  contents  of  interviews  and  reflections  according  to  recurrent  themes,  identified 
change  indicators,  and  came  together  to  achieve  consensus.  We  transferred  the  contents  of  students’  HyperStudio 
portfolios  to  a table  with  columns  of  text,  images,  and  researcher  comments.  In  the  comments  column,  the 
researchers  recorded  impressions  regarding  indicators  of  achievement,  self-esteem,  and  multicultural  issues. 
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Results 

We  found  that  students’  self-esteem,  academic  achievement,  and  multicultural  understanding  increased  in 
response  to  Cultural  Connections.  Four  overarching  themes  emerged  from  the  data:  growth,  empowerment,  comfort 
with  technology,  and  mentorship.  These  themes  permeated  each  data  source  and  applied  to  all  participants  of  the 
project.  Participants  grew  personally  and  intellectually.  They  felt  empowered  to  achieve  goals.  They  became 
comfortable  with  technology,  and  they  provided  and/or  received  mentorship.  Students  mentored  and  learned  from 
each  other.  They  also  had  the  benefit  of  receiving  mentorship  from  both  their  local  and  distant  teachers.  Teachers, 
administrators,  and  researchers  also  learned  from  each  other  as  we  shared  understandings. 

S'Mdcnts’  increased  self-esteem  was  indicated  by  their  heightened  poise,  communication,  and  leadership 
skills.  One  student  wrote,  “I’m  learning  to  be  more  open,  how  to  express  my  thoughts  and  ideas,  and  be  clear  in  my 
speech1'  (SI/S4/BC).  At  the  beginning  of  the  school  year  students  had  used  technology  for  little  more  than  drill  and 
practice.  By  the  end  of  the  school  year  they  described  themselves  as  technologically  proficient  in  multimedia 
development  and  telecommunications,  a reflection  that  their  teachers  corroborated.  One  student  won  the  district 
wide  multimedia  development  award. 

Language  in  the  pre-project  and  post-project  interview’s  and  reflections  indicated  transformation  from  self- 
bclittlemcnt  to  pride.  For  instance,  early  in  the  project  several  students  compared  themselves  unfavorably  to  their 
distant  partners.  One  student  said,  “You  sec  them  and  it’s  like  they  have  no  problems  at  all”  (SI/S5/BC).  Another 
wrote  that  “Jones  students  are  not  nervous  in  front  of  the  camera  and  we  are”  (SI/S7/BC).  By  the  same  token,  Jones 
students  indicated  that  Berta  Cabaza  students  seemed  more  poised  and  less  nervous.  By  the  end  of  the  school  year 
students  remarked  that  they  were  no  longer  nervous  in  front  of  a group  or  a camera.  A representative  comment  in 
response  to  the  question,  “What  are  you  learning  from  Cultural  Connections,”  follows:  “I  am  learning  more  about 
myself  and  being  more  aware  of  what  and  who  I am.  Also  [I  am]  learning  to  be  grateful  for  what  I have  and  not  be 
shy  or  nervous.”  She  also  learned  to  value  a distant  student  partner  on  an  activity;  “We  arc  friendly  to  each  other. 
Both  of  us  are  creative”  (SR/S7/BC), 

Students'  increased  academic  achievement  was  indicated  by  comparing  scores  on  the  benchmark  reading 
tests  administered  at  Berta  Cabaza  Middle  School.  Test  scores  in  November  indicated  that  students  in  the  San  Benito 
teacher’s  four  classes  had  mastered  the  same  number  of  reading  objectives  prior  to  initiation  of  Cultural 
Connections.  We  identify  the  four  classes  as  DL  for  distance  learning,  class  1,  class  2,  and  class  3.  The  DL  group 
met  several  times  with  their  distant  partners  before  the  second  school  benchmarking  test  was  administered  in 
January.  The  DL  class’s  mean  scores  improved  by  27%.  Class  1 scores  declined  by  4%,  class  2 scores  increased  by 
3%,  and  class  4 increased  by  14%.  The  DL  class  continued  to  meet  with  their  distant  partners  prior  to  a third  testing. 
When  compared  with  means  in  November,  the  third  benchmarking  test  administered  in  February  showed  that 
students  in  DL  improved  by  3%,  class  1 scores  declined  by  9%,  class  2 scores  declined  by  7%,  and  class  4 declined 
by  2%  (sec  Figure  1). 
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Figure  I.  Mastery  of  reading  objectives  at  Berta  Cabaza  Middle  School  in  the  distance  learning  class  ( n = J4), 
class  ! (n  =22),  class  2 (n  = 24),  and  class  3 (n  = 24). 


In  addition  to  increased  reading  scores,  portfolio  analysis  revealed  that  by  the  end  of  the  year,  1 1 of  the  14 
Berta  Cabaza  students  in  distance  learning  intended  to  go  to  college  and  they  all  expressed  interest  in  a successful 
career.  One  student  wrote,  “My  main  goal  is  to  graduate  from  college.  I want  to  be  the  first  from  my  family  to  reach 
this  goal.  I really  have  confidence  in  myself  and  I believe  I can  do  it.  After  graduating  from  college,  I am  hoping  to 
get  a job  as  a physical  therapist”  (P/S3/BC). 

Students’  increased  multicultural  understanding  was  indicated  by  their  comments  regarding  themselves  and 
others  in  the  project.  The  open-ended  question,  “What  are  you  learning  from  meeting  with  your  distant  partners  for 
videoconferences?”  elicited  responses  such  as,  ‘Tm  learning  that  all  people  have  different  kinds  of  taste  and 
different  goals”  (SR/S8/BC),  “I  am  learning  what  they  like  to  do  and  how  they  feel  about  things.  I am  learning  that 
they  have  trust  to  show  us  their  personal  things”  (SR/S2/BC),  and  “I  am  learning  more  ideas,  new  ideas,  and  old 
ideas.  I am  learning  that  many  people  have  great  ideas”  (SR/S7/BC). 

Language  regarding  distant  others  changed  over  time.  Early  in  the  project  students  spoke  of  their  distant 
partners  in  generalized  terms  and  emphasized  differences.  For  instance,  one  student  commented,  “We’re  eighth 
graders  and  they’re  fifth  graders.  They  speak  differently  and  arc  different  colors  and  some  of  their  last  names  are 
weird”  (SI/S6/BC).  Later,  their  language  was  more  personalized  and  comments  focused  on  similarities:  “We  like  the 
same  rnusica  and  we  have  almost  the  same  classes,  but  in  different  cities.  We  have  the  same  taste  in  some  food, 
music,  and  sports,  and  I say  we  all  liked  distance  learning”  (SR/S4/BC);  and  “Chris  likes  to  play  around  and  I do 
too.  He  is  very  funny  and  he  is  a student  like  me”  (SR/S3/BC). 

After  the  first  meeting  in  November  a young  man  responded  to  the  question  of  what  was  learned  from  the 
distance  collaboration  by  commenting  that  “Ms.  Davis’s  students  don’t  seem  to  be  afraid  of  the  camera,  they  are  not 
afraid  to  talk”  (Sl/Sl/BC).  The  statement  is  impersonal  regarding  The  distant  others  and  the  student  paints  himself  in 
a negative  light  compared  to  them.  In  May  he  responded  to  the  same  question  by  writing,  “That  they  learn  from  us 
and  we  learn  from  them.  Also  they  are  very  smart  kids  and  I’m  learning  that  I should  change  my  attitude” 
(SR/S3/BC).  This  statement  is  more  personal  and  illustrates  both  self-respect  and  respect  for  distant  others. 

Portfolios  revealed  expanded  multicultural  understanding  over  time.  In  one  student’s  HyperStudio™  stack, 
after  describing  his  family,  house,  and  state,  he  included  Earth  as  the  larger  context  of  his  home.  He  wrote  “it 
[Earth]  has  different  climates,  different  geographic  figures,  and  different  people  as  in  culture.”  We  interpreted  this 
broad  and  inclusive  sense  of  home  as  reflecting  a vision  of  world  community,  and  therefore  multiculturalism. 
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Educational  Implications 

Dewey  (1938)  defined  growth  as  the  ability  to  secure  meaning  from  experience  and  to  act  in  ways 
instrumental  to  the  achievement  of  worthwhile  ends.  In  Cultural  Connections,  diverse  students  across  Texas 
collaborated  on  multicultural  activities  which  helped  them  grow  in  self-esteem,  achievement,  and  multicultural 
understanding.  Cross-classroom  collaboration  made  it  possible  for  young  adolescents  to  expand  their  world-views  in 
preparation  for  contributing  in  our  increasingly  multicultural  environment.  This  project  demonstrates  that  in 
networked  classrooms  students  can  connect  with  distant  others  to  learn  about  and  from  their  perspectives  and  to 
increase  their  multicultural  understandings.  In  addition,  distance  technologies  can  foster  team  teaching  across 
cultures  and  geographical  distances. 
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Prediction  of  and  Differences  in  Computer  Use  when  Universally  Available 
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Problem 

Expectations  for  faculty  members  to  integrate  educational  computing  into  undergraduate  teaching  arc  high. 
Levels  of  integration  among  faculty  arc  often  low.  The  universal  availability  of  computers  through  the  adoption  of 
notebook  computer  for  faculty  by  the  University  may  be  one  answer.  The  researcher  studied  this  problem  through  a 
survey  of  faculty  at  three  undergraduate  institutions  which  adopted  notebook  computers  for  their  faculty  and 
students. 

Participants 

Longitudinal  Study 

1 . Undergraduate  faculty  from  three  small  campuses  (n  = 85). 

• Valley  City  State  University  (VCSU)  (n  = 36) 

• Mayville  State  University  (MaSU)  (n  = 28) 

• Jamestown  College  (JC)  (n  = 2 1 ) 

2.  Respondents  answered  two  questionnaires  one  year  apart 

3.  Age,  Academic  Rank,  Innovativeness  Scale,  and  Subjective  Norms  Index  determined  prior  to  adoption  of 
notebook  computers  on  any  of  the  campuses. 

4.  The  second  questionnaire  was  used  to  determine  level  of  computer  use. 

5.  Faculty  had  had  notebooks  between  six  months  and  one  year  at  the  time  of  the  second  questionnaire.  The  three 
campuses  had  differing  infrastructures  and  expectations 

6.  The  matched  responses  from  the  two  surveys  were  used  to  determine  changes  on  the  three  campuses. 

Variables 


Dependent  Independent  Demographics 

Phase  I Levels  of  Computer  Use  Subjective  Norms  Age 

Innovativeness  Academic  Rank 

Piiasc  II  University  Level  of  Computer  Use 

Faculty  Stages  of  Concern 
Student  Technology  use 
Faculty  Technology  Use 
Computer  Anxiety 
Frequency  of  Software  Use  by 

Faculty 


11 1 


107 


Questions 

1 . What  variables  might  predict  computer  use  by  faculty? 

2.  •Arc  there  differences  in  the  following  variables  because  of  the  differences  in  the  institution  and  arc  there 
differences  over  time? 

1 . What  is  the  level  of  computer  use  among  faculty  at  the  three  institutions? 

2.  What  happens  to  the  use  of  computer  technology  when  notebook  computers  arc  available  to  faculty  and  or 
students? 

3.  Does  the  faculty’s  computer  anxiety  become  lower  after  faculty  have  had  access  to  the  notebooks  for  a 
period  of  time? 

4.  Does  the  faculty  and  student  access  to  the  notebook  computers  change  the  types  of  concerns  expressed  in 
the  Stages  of  Concerns  Questionnaire  (SoCQ)? 

Instruments 

• Levels  of  Computer  Use  scale  (LCU)  (Marcinkicwicz  & Wclliver) 

• Subjective  Norms  Seale  (Mareinkiewicz  & Regstad) 

• Innovativeness  Scale  (IS)  (Hurt,  Joseph,  & Cook) 

• Computer  Anxiety  Index  (CAIN)  (Simonson,  Maurer,  Montag-Torardi  & Whitaker) 

• Stages  of  Concerns  Questionnaire  (Hall,  George  & Rutherford) 

Procedures 

Participants  completed  a Computer  Technology  in  Teaching  questionnaire  composed  of  the  above  measures 
as  well  as  demographic  and  computer  related  questions.  Faculty  received  the  questionnaire  prior  to  the  adoption  of 
notebook  computers  on  three  undergraduate  campuses.  The  faculty  received  the  same  questionnaire  one  year  later. 
During  the  interim  between  the  questionnaires,  the  three  campuses  adopted  notebook  computers  for  all  faculty  at 
varying  times.  One  of  the  campuses  also  adopted  notebooks  for  all  students. 

In  the  first  phase  of  the  study  the  dependent  variable,  level  of  computer  use,  was  determined  using 
responses  to  the  second  questionnaire.  The  independent  variables;  age,  academic  rank,  innovativeness  and  subjective 
norms  were  drawn  from  responses  to  the  first  questionnaire. 

The  second  phase  of  the  study  dealt  with  technology  teaching  changes  on  the  three  campuses.  Indicators  of 
this  change  included:  computer  anxiety,  faculty  and  student  use  of  technology,  frequency  of  software  use  by  faculty 
and  level  of  computer  use.  Data  from  the  first  questionnaire  was  compared  to  data  from  the  second  questionnaire 
using  a linear  model  with  matched  pairs.  Only  data  from  faculty  who  responded  to  both  questionnaires  were  used. 

Analysis 

In  the  first  phase  of  the  study  the  LCU  scale  determined  the  criterion  variable,  level  of  computer  use.  The 
predictor  variables,  age,  academic  rank,  subjective  norms  and  innovativeness,  were  compared  to  the  level  of 
computer  use.  The  researcher  applied  a linear  regression  to  the  data.  Another  linear  model  was  then  utilized  to 
determine  the  effect  of  university  on  the  level  of  computer  use. 

In  the  second  phase  of  the  study,  a linear  model  was  also  used  to  compare  each  of  variables.  An  F test 
indicated  differences  in  the  variables  among  the  campuses  and  differences  over  time.  T- tests  were  done  between  the 
campuses  to  determine  which  of  the  campuses  was  responsible  for  the  difference. 

Results 

In  the  first  phase  of  the  study,  the  data  indicated  that  one  variable—  Subjective  Norms  Index-  was  a 
significant  predictor  of  the  faculty’s  levels  of  computer  use.  Two  of  the  remaining  variables,  Age  and 
Innovativeness  Scale,  while  were  somewhat  predictive.  The  last  variable.  Academic  Rank  did  not  add  to  the 
predictive  value  beyond  that  indicated  by  Subjective  Norms  Index,  Innovativeness  Scale  and  Age.  The  researcher 
applied  a fixed  effect  for  university  to  the  data  and  the  predictive  value  of  Subjective  Norms  Index  was  no  longer 
significant.  The  institution  at  which  the  faculty  was  employed  had  more  of  an  effect  on  computer  use  than  did  the 
criterion  variables. 


Table  1 Predictors  of  Computer  Use 


Variable 

t-lcst  statistic 

Sig.  WO/Efiecls 

Sig.  W/Effects 

Age 

-.567 

.057 

.093 

Innovativeness 

1.952 

.055 

.135 

Academic  Rank 

-.131 

.896 

.541 

Subjective  Norms 

2.555 

.013 

* 

.295 

*p<  .05 

In  ihc  second  phase  of  Ihc  study,  the  matched  data  indicated  significant  differences  in  the  campuses  over 
time  for  several  variables  including;  number  of  different  technology  uses  by  faculty  and  students  and  the  frequency 
of  software  use  by  faculty.  Only  one  variable,  number  of  student  required  uses  of  technology,  indicated  a difference 
among  the  campuses  as  a whole.  The  data  showed  significant  increases  in  level  of  computer  use,  number  of  faculty 
and  student  technology  uses  and  amount  of  software  use  occurred  with  the  VCSU  faculty  compared  to  the  rest  of  the 
respondents.  VCSU  was  also  the  only  campus  which  had  adopted  computers  for  students  during  the  period  of  the 
study.  See  Table  II. 


Table  II.  - Differences  Among  Campuses  and  Between  Pre 

and  Post  Scores 

Among  all 

In  campuses 

In  VCSU 

Differences 

Between 

Between  VCSU 

Between  VCSU 

campuses 

over  lime 

over  lime 

MaSU&  JC 

& MaSU 

& JC 

(F  test) 

(F  lest) 

(t-tesl) 

(l-lcsl) 

(blest) 

(l-lcsl) 

Variable 

Computer 

.822 

.188 

.617 

.106 

.1 19 

.771 

Anxiety 

Level  of  Use 

.430 

.092 

.048 

* 

.324 

.818 

.205 

# of  Faculty 

.804 

.000 

.000 

.743 

.511 

.791 

Tech.  Uses 
# of  Student 

.018 

* }:* 
.000 

.000 

.571 

.008 

.063 

Tech.  Uses 
Amount  of 
Faculty 

* 

.161 

*!j<* 

.001 

ijs  H< 
.022 

.058 

253 

.315 

Software  use 

** 

*p<  .05 

* 

**p<  .01 

***p  <.001 

For  more  information  visit  this  web  sight:  http://www.vcsu.nodak.edu/offices/lillciii/Idnks.hlm 


Stages  of  Concerns 

The  following  interpretation  is  one  part  of  the  research  completed  on  a problem  studied  through  a survey  of 
faculty  employed  at  three  undergraduate  institutions  which  adopted  notebook  computers  for  their  faculty. 

Interpretation  - Questionnaire  I 

Interpreting  the  Stages  of  Concerns  Questionnaire  (SoCQ)  involved  determining  the  mean  of  each  of  the 
seven  concerns  and  comparing  the  changes  in  those  means  from  1996-1997  for  each  of  the  concerns.  Charts 
detailing  levels  of  each  concern  are  available  at  this  v/cb  site  http://www.vcsu.nodak.edu/offices/tilleiii/Links.him. 
The  following  narrative  was  written  using  profile  interpretations  from  the  Measuring  Stages  of  Concern  about  the 
Innovation  Manual , Hall,  George  and  Rutherford. 

When  the  SoCQ  was  recorded  in  1996,  prior  to  the  adoption  of  the  notebook  computers,  faculty  from  both 
VCSU  and  JC  had  very  similar  patterns  of  concern  about  computer  technology.  Both  groups  indicated  they  knew 
quiet  a lot  about  computers  and  were  not  threatened  by  them.  They  also  had  minimal  to  no  concerns  about  managing 
their  use  but  some  concern  about  the  consequences  of  use  for  students. 
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MaSU  faculty  inciicalcd  significantly  different  concerns  including:  wanting  more  information  about  the 
computers,  intense  personal  concern  about  computers  and  their  consequences  for  them,  no  concerns  about  the 
relationship  of  students  to  use  and  they  were  more  likely  to  be  negative  toward  the  innovation. 

Interpretation  - Questionnaire  II 

At  the  time  of  the  second  questionnaire  in  1997  the  patterns  had  changed,  some  significantly.  JC’s  pattern 
remained  nearly  the  same  as  the  previous  year.  MaSU  had  moved  into  a pattern  nearly  identical  to  that  of  VCSU’s 
and  JC’s  from  the  previous  year.  MaSU’s  greatest  concerns  were  about  looking  for  ideas  from  others,  reflecting 
more  a desire  to  learn  from  what  others  know  and  arc  doing,  rather  than  concern  for  collaboration.  VCSU,  in  the 
second  questionnaire,  showed  significantly  different  concerns  including,  a fairly  intense  involvement  with 
computers  and  concerns  about  a collaborative  effort  in  relation  to  the  other  high  stage  concerns  Also  some 
individuals  indicated  that  they  already  know  all  about  computers  and  have  plenty  of  ideas. 

Discussion 

The  results  of  the  concerns  may  he  explained  by  the  difference  in  the  culture  on  the  three  campuses. 
Information  from  the  questionnaire  indicated  several  differences  on  the  campuses.  First,  at  the  time  of  the  first 
questionnaire  in  the  Spring  of  1996  faculty  on  the  VCSU  and  JC  campuses  had  significantly  more  experience  with 
computers  than  did  faculty  on  the  MaSU  campus.  One  year  later  the  difference  was  no  longer  significant,  probably 
due  to  the  disappearance  of  the  non-user  on  the  MaSU  campus.  See  Table  I. 


Table  III.  - Use  Level  on  Campuses 


# 1996 

Mean  of 

# 1997 

Mean  of 

Computer 

Computer  Use 

Institution 

Use  1996 

1997 

MaSU 

23 

2.91 

25 

3.56 

JC 

19 

3.79 

20 

3.85 

VCSU 

35 

3.80 

33 

3.79 

Total 

77 

3.53 

78 

3.72 

Computer  Use  in  Years  0=  non-use 

1=  < 1 year  2 

= I -2  years  3 = 3-4 

years  4=  >5 

years 

Second,  the  results  of  the  Subjective  Norms  Survey  indicated  that  VCSU  faculty  scored  significantly  higher 
than  the  other  two  campuses.  This  indicates  the  faculty  on  the  VCSU  campus  perceive  the  expectations  of  students, 
peers,  and  administration  arc  more  important  than  did  other  faculty  in  the  study.  Sec  Table  II 


Table  IV.  - Differences  in  Subjective  Norms  Score  at  the  Three  Institutions 


Subjective 

Norms 

# of 

MaSU 

JC 

VCSU 

Institution 

Mean 

Faculty 

Significance 

Significance 

Significance 

MaSU 

50.027 

26 

- 

- 

— 

JC 

50.472 

21 

.004 

- 

- 

VCSU 

56.075 

37 

.001 

.004 

— 

Mean/  Total 

52.802 

84 

Third,  during  the  period  of  the  study  only  VCSU  adopted  notebooks  for  their  students.  Also,  the 
infrastructure  of  the  campus  included  many  rooms  capable  of  multimedia  (computer)  projection  and  Internet 
connections  by  students.  The  MaSU  campus  had  plans,  the  year  following  the  study,  to  adopt  notebooks  for  their 
students  with  the  same  infrastructure  changes  as  VCSU  however,  JC  had  no  multimedia  capable  rooms  and  had  no 
plans  for  student  adoption. 

Length  of  computer  use,  perceived  importance  of  student,  peer  and  administration  expectations  of  use  and 
student  adoption  arc  thought  to  be  responsible  for  the  cultural  differences  on  the  campuses.  The  differing  cultures 
arc  thought  to  have  influenced  the  responses  to  the  SoCQ  over  the  one  year  period  of  the  study. 
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Abstract 

The  aim  of  this  study  was  to  investigate  some  of  the  learner  variables  that  may  have  an  influence  oil  the 
University  academic  achievement  in  a distance-  versus  a conventional  education  setting. 

Descriptive  and  analytic  statistics  were  used  to  analyze  the  data  of  this  study  by  using  "Pearson  r'\  on  the 
one  he . d,  and  " F-test ",  on  the  other.  They  revealed  the  following  results: 

1-  The  university  academic  achievement  at  both  types  of  education  - distance  and  conventional  - have  been  affected 
significantly  by  similar  variables.  They  were: 

a)  Prior  high  school  academic  achievement  (It  has  a significant  positive  correlation  with  the  University 
academic  achievement), 

b)  Prior  high  school  specialization  (It  was  in  favor  of  science  specialization.),  and 

c)  University  specialization  (It  was  also  in  favor  of  science  specialization). 

The  gender  factor  has  just  affected  significantly  the  university  academic  achievement  in  a distance 
education  setting  ( It  was  in  favor  of  females). 

2-  But  the  University  academic  achievement  in  both  types  of  education  have  npl  been  affected  significantly  by  these 
variables: 

a)  the  locus  of  control, 

b)  work  responsibility, 

c)  and  the  university  academic  level. 

According  to  these  results,  the  researcher  recommends  the  university  curriculum  planners  consider  courses 
in  science  to  be  compulsory  courses  regardless  of  student's  specialization.  Such  courses  arc  expected  to  force 
students  to  use  their  mental  process  deeply  during  learning,  hence,  to  enhance  their  academic  achievement. 

Further  analysis  was  done  in  this  research , thus  more  recommendations  were  suggested. 

Introduction 

At  any  university,  administrators  always  try  to  maintain  a high  quality  of  education.  One  measure  which  is 
still  commonly  used  to  asses  such  quality  is  academic  achievement. 

University  academic  achievement  cither  in  a distance  or  a conventional  education  setting  is  affected  mainly 
by  two  groups  of  variables  as  classified  by  "Robert  Gagne".  They  are:  1)  the  internal  variables,  and  2)  the  external 
variables  (See,  Gagne,  1977;  Gagne,  Briggs,  & Wager,  1992). 

The  internal  variables  arc  theoretically  related  to  the  learner's  characteristics  such  as  intelligence,  prior 
achievement,  prior  knowledge,  aptitudes,  attitudes,  motivation,  locus  of  control,  field  dependence  and  field 
independence,  learning  style,  work  responsibility,  socioeconomic  class,  etc.;  Whereas  the  external  variables  arc 
basically  related  to  the  environment  in  which  learning  takes  place,  such  as  the  physical  conditions  (i.c., 
spaces/buildings,  halls,  studios,  galleries,  auditoriums),  teacher's  competency,  delivery  systems,  administration 
system,  planning  and  designing  system,  curriculum,  mass  media,  educational  aids,  budget,  etc.,  ( e.g.,  Barry,  1992; 
Kaye,  and  Rumble,  1991;  May  ton,  1989;  Rczabek,  1992;  Sammons,  1988;  Verduin  & Clark,  1991)  (Sec  Figure  1). 
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Considering  that  distance  education  is  altogether  a different  system  with  more  heterogeneous  learner 
characteristics,  and  a different  delivery  system  from  the  conventional  one  (Wilson,  1991),  we  assume  that  the 
academic  achievement  in  a distance  education  setting  will  be  affected  differently  compared  to  that  of  education  in  a 
conventional  one. 

Some  previous  studies  tested  this  hypothesis  and  investigated  the  effectiveness  of  distance  versus 
conventional  education  in  terms  of  students’  academic  achievement  and  their  attitudes  towards  learning.  Ritchie  and 
Newby  (1989),  for  example,  compared  two  groups  of  undergraduates:  one  group  studied  a course  in  a conventional 
education  setting,  via  lecture,  and  the  other  group  studied  the  same  course  in  a distance  education  setting,  via 
television.  They  found  that  the  first  group  of  conventional  education  had  better  achievement  than  the  second  group 
of  distance  education.  They  also  found  that  the  students  in  the  conventional  education  setting  enjoyed  the  course 
more  than  their  counterparts  who  learned  at  a distance,  via  television. 
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Smith  and  McNelis  (1993)  supported  (he  above  results  when  they  look  a class  of  16  graduates  who  were 
studying  a requirement  course  in  a distance  setting  and  compared  them  with  the  same  level  class  consisting  of  12 
graduates  studying  the  same  course  in  a conventional  education  setting.  At  the  end  of  the  course,  the  assessment  of 
academic  achievement  showed  that  the  performance  of  students  in  a distance  education  setting  was  significantly 
lower  than  that  of  their  counterparts  in  a conventional  education  selling.  In  addition,  the  altitudes  of  distance 
education  students  toward  learning  were  negative. 

In  contrast,  Chu  and  Schram  (1976)  found  that  students  who  learned  by  television  at  a distance  performed 
significantly  better  than  their  counterparts  who  learned  the  same  course  by  lecture  in  a conventional  education 
setting.  They  also  found  that  students,  at  a distance,  especially  the  young  students,  preferred  television  to  lecture  as 
a tool  of  instruction.  Horowita  ( i 979)  also  found  that  students'  altitude  toward  distance  education  was  more  positive 
than  toward  the  conventional  one  (See,  Feasicy,  1982). 

On  the  other  hand,  Haynes  & Dillon  (1992)  failed  to  find  any  significant  differences  >n  the  academic 
achievement  of  students  who  studied  at  a distance  and  of  their  counterparts  who  studied  in  a conventional  education 
setting,  or  in  their  attitudes  towards  learning. 

By  looking  at  another  variable.  Field  Dependent-  versus  Field  Independent  learning,  the  literature  generally 
indicated  that  most  of  distance  education  learners  were  field  independents  and  had  a preference  for  the  solitary 
situations  and  self-defined  goals,  strategies  and  reinforcement,  and  were  associated  with  the  idea  of  self-directness, 
and  autonomy;  whereas  most  of  the  conventional  education  learners  were  field  dependents  who  had  a preference  for 
the  external  controls  and  more  structured  environment  (See,  Rotter,  1966;  Wilson,  1991,  p.43). 

Wilson  (Ibid.)  had  also  summarized  the  results  of  several  research  projects  related  to  other  characteristics 
of  distance  learners  compared  to  conventional  ones.  He  found  that  distance  learners  were  generally  older  than  the 
conventional  learners,  and  their  age  ranged  between  20-40,  at  the  average  of  37  years.  Most  of  them  were 
professional  people  who  went  to  school  part-time;  66%  of  them  were  employed  full-time.  Most  of  them  were 
women;  however,  this  seemed  to  be  a trend  only  in  more  developed  countries.  Wilson  also  found  that  84%  of 
distance  learners  were  white,  and  75.5%  were  married.  He  found  that  41%  of  distance  learners  were  at  a distance 
because  going  to  on-campus  classes  conflicted  with  their  work,  or  conflicted  with  their  leisure  time  (12%),  and  51% 
of  them  chose  distance  education  to  minimize  travel,  and  others  because  of  social,  economic  or  geographical 
reasons. 

From  the  above  results,  we  can  say  that  the  characteristics  of  learners  arc  clearly  different  in  a distance- 
compared  to  a conventional  education  setting;  hence,  the  academic  achievement  varies  from  one  situation  to  another. 
Since  we,  in  Palestine,  arc  experiencing  a new  system  of  education,  distance  education,  and  arc  careful  to  secure  a 
high  quality  of  education  in  this  new  system,  the  author  of  this  study  intended  to  investigate  some  of  the  learner 
variables  that  might  affect  the  academic  achievement  at  a distance-compared  to  a conventional  education  selling. 
The  variables  investigated  in  this  study  were:  the  learner's  gender,  locus  of  control,  prior  high  school  academic 
achievement,  prior  high  school  specialization,  university  specialization,  university  academic  level,  and  work 
responsibility  (Sec,  Figure  2). 


Therefore,  the  aim  of  this  study  was  to  investigate  the  relationship  between  learner  variables  and  university 
academic  achievement  at  a distance  education  university  (Al-Quds  Open  University)  and  a conventional  education 
university  (An-Najah  National  University). 

Importance  of  research 

As  mentioned  above,  wc,  in  Palestine,  arc  experiencing  the  distance  education  of  Al-Quds  Open 
University,  and  arc  careful  to  establish  a high-quality  education;  therefore,  wc  believe  that  conducting  such  research 
will  help  us-  as  instructional  designers,  to  obtain  information  about  our  learners'  charactcristics/variablcs  that  might 
affect  their  academic  achievement,  and  in  what  manner,  hence,  to  use  this  information  as  a basis  for  modifying  the 
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current  practice,  such  as  how  to  modify  instructional  materials  and  teaching  methods  to  accommodate  the  learners’ 
characteristics.  This  will,  consequently,  help  us  to  provide  high  quality  education,  and  to  achieve  higher  educational 
outcomes.  We  also  hope  to  draw  a model  of  interaction  between  learner  variables  and  university  academic 
achievement,  and  broaden  it  depending  on  the  findings. 

Methodology 

Sample: 

A random  sample  of  250  male  and  female  students  was  taken  from  Al-Quds  Open  University  branches 
located  in  three  districts  of  the  northern  part  of  Palestine.  It  came  arbitrarily  as  follows:  Nablus  (108),  ToulKarcin 
(67),  and  Jenin  (75).  In  contrast,  a random  sample  of  250  male  and  female  students  was  taken  from  An-Najah 
National  University  (conventional  education)  which  matched  the  Al-Quds  Open  University  learners  in  their 
specialization  (Literature,  Science,  Education,  and  Business  Administration),  and  academic  level  (freshmen, 
sophomores,  juniors,  and  seniors). 

Procedure  and  Measures: 

Rotter's  Locus  of  Control  Scale,  plus  a questionnaire  which  identified  some  of  the  learners’  variables,  were 
distributed  to  collect  data  from  the  samples  drawn  randomly  from  the  registered  students,  (Fall  semester, 
1996/1997),  from  both  Universities:  Al-Quds  Open  University  (250  students),  and  An-Najah  National  University 
(250  students).  The  data  collected  were  related  to  learners’  gender,  locus  of  control,  the  average  of  government 
general  high  school  exam  (GGHSE),  prior  high  school  specialization  (Literature,  Science),  university  specialization 
(Literature,  Science,  Education,  and  Business  Administration),  University  academic  level  (freshmen,  sophomores, 
juniors,  and  seniors),  and  work  responsibility  (work,  no  work)  (See,  Figure  2 again).  The  returned  questionnaires 
were  (221)  from  Al-Quds  Open  University  and  (203)  from  An-Najah  National  University. 

The  results  were  compared  to  the  reported  university  academic  achievement  which  was  expressed  in  the 
cumulative  average  obtained  up  to  the  Fall  semester  96/97  in  both  types  of  University  to  see  if  there  were  a positive 
and  significant  relation  between  the  learner  variables  and  university  academic  achievement. 


Results 

A correlation  coefficient  by  using  (pearson  r),  and  one-way  analysis  of  variance  by  using  "F-tesf,  have 
revealed  the  following  results: 

First:  Al-Quds  Open  University  (distance  education): 

The  academic  achievement  at  Al-Quds  Open  University  was  significantly  affected  by  the  following  factors: 

1 - Gender.  It  was  in  favor  of  females. 

2-  Prior  high  school  academic  achievement.  The  correlation  between  the  average  of  government  general  high  school 
exam  and  the  University  academic  achievement  was  (r=.36,  pc.Ol). 

3-  Prior  high  school  specialization.  It  was  in  favor  of  science  specialization  . 

4-  University  specialization.  It  was  in  favor  of  science  and  literature  specializations. 

The  academic  achievement  at  Al-Quds  Open  University,  however,  was  not  affected  significantly  by  the 
following  factors: 

1 - work  responsibility, 

2-  Locus  of  control,  or 

3-  University  academic  level. 

Second:  An-Najah  National  University  (conventional  education): 

The  academic  achievement  at  An-Najah  National  University  was  significantly  affected  by  the  following 

factors: 

1-  Prior  high  school  academic  achievement.  The  correlation  between  the  average  of  government  general  high  school 
exam  and  the  University  academic  achievement  war  (r=.27,  p<  .01 ). 


2-  Prior  high  school  specialization.  It  was  in  favor  of  science  specialization. 

3-  University  specialization.  It  was  also  in  favor  of  science  specialization. 

But  the  academic  achievement  at  An-Najah  National  University  was  no!  affected  significantly  by  the 
following  factors: 

1 - Gender, 

2-  work  responsibility, 

3-  Locus  of  control,  or 

4-  University  academic  level  (See,  Table  1 for  statistical  information). 

Third:  A comparison  between  the  results  of  Al-Quds  Open  University  and  An-Najah  National  University: 

By  doing  a comparison  between  the  results  of  both  types  of  University:  distance  education  versus 
conventional,  we  reached  the  following  results: 

1-  The  gender  factor  had  a significant  effect  on  the  university  academic  achievement  in  a distanee-but  not  in  a 
conventional  education  setting.  It  was  in  favor  of  females. 

2-  The  locus  of  control  factor  did  not  have  a significant  effect  on  the  university  academic  achievement  in  both 
settings  - distance  and  conventional  education  - but  it  was  found  that  students  of  distance  education  were  more 
internally  oriented(E-I  = 9.43)  than  their  counterparts  of  conventional  education  (E-I  = 1 1 .35). 

3-  Prior  high  school  academic  achievement,  which  was  expressed  in  GGHSE,  had  a significant  effect  on  the 
university  academic  achievement  in  both  settings:  distance  and  conventional  education  (r=.36  versus  r=.24,  at  P> 
.01). 

4-  Prior  high  school  academic  achievement,  which  was  expressed  in  GGHSE,  was  higher  for  students  in 
conventional  education  (82.2%)  than  their  counterparts  in  distance  education  (70.1%). 

5-  Prior  high  school  specialization  had  a significant  effect  on  the  university  academic  achievement  in  both  settings: 
distance  and  conventional  education.  The  significance  was  in  favor  of  science  specialization  in  both  types  of 
education. 

6-  The  university  specialization  had  a significant  effect  on  the  university  academic  achievement  in  both  settings: 
distance  and  conventional  education.  The  significance  was  in  favor  of  science  specialization  in  conventional 
education,  and  it  was  in  favor  of  science  and  literature  in  distance  education. 

7-  Each  of  the  following  factors  had  not  had  a significant  effect  on  the  university  academic  achievement  in  both 
educational  settings.  They  arc  the  locus  of  control,  work  responsibility,  and  university  academic  level  (See,  Table  1 
again). 

8-  University  academic  achievement,  which  was  expressed  in  the  University  cumulative  average,  was  higher  for 
conventional  education  students  (77.2%)  than  for  their  counterparts  in  distance  education  (74.6%). 

9-  Based  on  the  sample  of  this  study,  the  number  of  students  who  held  work  responsibility  was  higher  in  distance 
education  (53)  than  in  conventional  one  (12).  But  the  number  of  students  in  both  settings  distance  and  conventional 
education  who  did  not  work  during  their  study  was  higher  (296)  than  the  number  of  students  who  did  work  (65). 
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Discussion  and  Recommendations 

II  seems  that  the  most  important  result  of  this  study  was  that  academic  achievement  in  both  types  of 
university,  open  and  conventional  one,  was  affected  similarly  by  the  same  variables  of  learner,  with  a slight 
difference.  This  is  true  at  least  in  the  Palestinian  case.  We  could  say  according  to  the  findings  of  this  research  that 
the  learners  who  had  high  grades  in  high  school  with  a scientific  background  arc  the  most  successful  students  in  both 
types  of  University.  Even  at  the  University  levels,  learners  who  specialized  in  science  achieved  higher  grades  than 
those  who  specialized  in  education,  business  administration,  or  literature.  While  it  is  impossible  to  direct  all 
students  to  specialize  in  science  for  many  reasons  including  the  integration  of  the  society,  it  is  worthwhile  to  modify 
our  curriculum,  either  at  high  school  or  university  level,  to  include  more  topics  in  science  besides  other  topics.  It 
seems  that  the  topics  of  science  with  all  symbols,  formulas,  and  logical  issues  have  a positive  effect  in  activating 
deep  mental  processes  than  the  other  topics,  thus  increasing  students'  learning  achievement.  Well,  more  research  is 
recommended  on  this  issue  to  confirm  such  a result. 

The  interesting  result  is  that  the  gender  factor  was  affecting  university  academic  achievement  just  in  a 
distance  education  setting  not  in  a conventional  one.  It  was  in  favor  of  females.  This  result  was  supported  by 
Darwazeh  and  Abu-Amsheh  (1993)  when  they  found  that  the  academic  achievement  of  females  was  higher  than 
males  who  study  in  Al-Quds  Open  University  but  not  in  An-Najah  National  University,  the  conventional  one.  It 
seems  that  the  nature  of  the  distance  educational  system,  which  doesn’t  demand  that  their  learners  attend  the 
university  on  a systematic  schedule  helps  females  more  than  males  to  feel  free  in  organizing  their  schedule  at  their 
convenience,  so  they  can  manage  the  house  responsibility  and  their  studies  at  the  same  time.  This  could  be  true 
especially  for  married  females  who  hold  responsibilities  for  their  homes  and  families.  However,  more  research  is 
recommended  on  this  issue. 

With  respect  to  the  prior  high  school  academic  achievement  which  is  expressed  in  GGHSE,  it  seems  that 
this  factor  is  a good  predictor  of  the  university  academic  achievement  either  in  the  Open  University  or  in  the 
conventional  one.  At  the  same  time,  it  does  not  hurt  to  use  other  factors  besides  the  GGHSE  to  predict  university 
academic  achievement,  such  as  the  university  entrance  exam,  or  ability  tests,  especially  in  the  conventional 
universities  which  require  their  applicants  to  meet  certain  conditions  in  order  to  be  university  candidates. 

With  respect  to  the  locus  of  control,  the  study  found  that  the  Open  University  learners  were  more  internally 
oriented  (E-I  = 9.43)  than  the  learners  of  the  conventional  one  (E-I  = 1 1.35).  This  result  docs  make  sense,  because 
most  of  previous  research  summarized  by  Wilson  (1991),  found  that  the  distance  learners  were  mainly  field 
independents  rather  than  field  dependents.  This  means,  to  some  extent,  that  they  are  self-directed  learners  and  have 
control  over  their  learning  goals,  strategies,  and  structure.  Well,  this  kind  of  result  needs  to  be  tested  further. 

Looking  at  the  result  which  indicated  that  the  academic  achievement  at  An-Najah  National  University  was 
higher  (x=  77.2%)  than  the  academic  achievement  at  Al-Quds  Open  University  (x=74.6%),  it  makes  sense  at  least  in 
Palestine,  because  the  conventional  universities  in  Palestine  demand  their  applicants  to  meet  certain  conditions  in 
order  to  be  university  candidates,  and  one  of  these  conditions  a high  average  of  GGHSE,  whereas  Al-Quds  Open 
university  does  not  demand  such  a condition.  Accordingly,  thi  ronventional  university  learners  are  considered  to  be 
elite  learners  from  the  beginning,  in  terms  of  their  academic  achievement,  whereas  it  is  not  the  case  in  Al-Quds 
Open  University.  But  the  surprising  result  is  that  the  academic  achievement  of  learners  who  entered  An-Najah 
National  University  with  a GGHSE  average  of  (82.2%)  has  been  declined  after  entering  the  University  and  became 
(77.2%),  whereas  the  academic  achievement  of  learners  who  entered  Al-Quds  Open  University  with  a GGHSE 
average  (70.1%)  has  been  enhanced  after  entering  the  University  (74.6%).  This  result  could  be  due  to  the  nature  of 
the  instructional  system  at  Al-Quds  Open  University  which  gives  learners  more  freedom  and  control  over  their 
learning,  whereas  the  nature  of  the  instructional  system  at  An-Najah  National  University  gives  teachers  more 
responsibility  and  control  over  learners'  learning.  It  is  well  known  in  education  that  the  learner-centered  system  is 
more  effective  than  the  tcachcr-centered  system  especially  at  higher  level  of  learning.  This  kind  of  observation 
leads  us  to  raise  a recommendation  to  the  conventional  university  administrators  in  Palestine  to  modify  their 
instructional  system  and  let  the  teachers  give  their  students  more  opportunities  to  get  involved  in  the  learning 
process,  thus  to  internet  effectively  in  the  instructional  process,  thar.  :s  now  the  case.. 

To  sum  up,  more  research  is  recommended  to  investigate  other  learner  variables,  such  as  academic  needs 
achievement,  learning  styles,  time  commitment,  perception  to  the  instructional  materials,  etc.,  and  to  investigate 
some  of  the  external  variables  related  to  the  instructional  environment  and  which  have  an  influence  on  students' 
academic  achievement. 
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The  Office  of  Technology  Assessment  (OTA;  1995)  recently  reported  that  U.  S.  schools  have  over  5.8 
million  computers  available  for  instruction.  Over  the  past  decade  the  number  of  students  per  computer  has  gone 
from  125  to  less  than  10  (Sandholtz,  Ringstaff,  & Dwyer,  1997).  Although  administrators  have  generally  welcomed 
new  technology  into  their  schools,  the  introduction  of  computers  into  individual  classrooms  has  created  challenges 
for  both  experienced  and  inexperienced  teachers  (Becker,  1993).  Many  common  barriers  to  technology  integration 
have  been  identified  including  limited  equipment,  training,  and  time  (Hadley  & Sheingold,  1993;  OTA,  1995; 
Ringstaff  & Yocam,  1994),  as  well  as  teachers'  resistance  to  change  and  their  current  pedagogical  beliefs  (Gilmore, 
1995;  Hancock  & Betts,  1994;  Hannafin  & Savenyc,  1993).  Brickncr  (1995)  suggested  that  these  obstacles  be 
categorized  as  first-  or  second-order  barriers  to  change,  defined  as  "the  extrinsic  and  intrinsic  factors  that  affect  a 
teacher's  innovation  implementation  efforts"  (p.  xvii).  First-order  barriers  arc  extrinsic  to  teachers  and  include  lack 
of  access  to  computers,  access  to  software,  time  to  plan  instruction,  and  technical  and  administrative  support. 
Second-order  barriers  are  intrinsic  to  teachers  and  include  beliefs  about  teaching,  beliefs  about  computers, 
organizational  context,  lack  of  instructional  models,  and  unwillingness  to  change.  While  many  of  the  first-order 
barriers  may  be  eliminated  through  the  procurement  of  additional  resources  and  the  use  of  traditional  training 
methods,  second-order  barriers  are  more  likely  to  persist,  requiring  different  resources  and  training  strategics. 

According  to  Sorgc,  Russell,  Mandell,  & Brickncr  (1995),  effective  technology  integration  training  should, 
among  other  things,  provide  assistance  on  "the  process  of  implementing  technology"  (1995,  p.  29,  emphasis  added). 
Although  the  implementation  stage  is  generally  recognized  as  being  key  to  effective  change  efforts  (Fullan  & 
Hargreaves,  1992;  Herriott  & Gross,  1979),  current  methods  of  teacher  training  often  fail  to  address  this  stage.  In 
many  instances,  "technology  staff  development  for  teachers  has  been  fragmented,  not  context-specific,  and  without 
ongoing  support"  (Meltzer  & Sherman,  1997,  p.  56). 

A body  of  literature  is  accumulating  that  suggests  that  changes  in  methods  of  staff  development  may  help 
alleviate  second-order  barriers  to  computer  integration  (Ringstaff,  Sandholtz  & Dwyer,  1991;  Ringstaff  & Yocam, 
1994;  Ritchie  & Wiburg,  1994).  A small  part  of  this  literature  advocates  the  use  of  students  as  an  integral  part  of  the 
technology  training  process.  Not  only  can  the  students  learn  computer  skills  alongside  their  teachers,  but  they  can 
provide  on-going  technical  and  emotional  support  as  the  teacher  begins  the  implementation  process  within  her 
curriculum.  The  Apple  Classroom  of  Tomorrow  (ACOT)  project  illustrates  how  using  student  technology  expertise 
in  the  classroom  may  effectively  address  implementation  barriers.  For  example,  Ringstaff  ct  al.  (1991)  stated  that 
“while  many  teachers  at  first  questioned  the  value  of  using  students  as  teachers  and  wondered  how  it  would  affect 
learning,  teachers  soon  realized  that  the  benefits  of  this  role  shift  went  far  beyond  saving  them  time"  (p.  12). 
Teachers  saw  change  not  only  within  their  students  but  within  themselves.  Teachers  often  took  the  role  of 
facilitators,  while  students  played  the  role  of  experts  (Ringstaff  & Yocam,  1994).  As  the  teachers  became  more 
confident  with  this  pedagogical  shift,  learning  to  integrate  technology  became  more  relevant.  “Only  classroom 
practice  (i.c.,  working  with  children)  provides  relevance  and  purpose  to  access  skills  and  information  and,  therefore, 
has  the  potential  for  long-term  changes  in  teacher  behaviors"  (Gilmore,  1995,  p.  254). 

Background  and  Purpose  of  the  Study 

In  the  fall  of  1995,  faculty  and  graduate  students  at  a mid  western  university  initiated  an  alternative 
approach  to  teacher  technology  development  at  a local  elementary  school  (referred  to  here  as  Midland  Elementary). 
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For  the  past  two  years,  all  students  (grades  1-5)  have  received  computer  training  at  least  one  day  a week  from  the 
media  specialist,  university  personnel,  and/or  volunteers,  An  additional  after-school  training  program  was  initialed 
in  the  fall  of  1996  to  train  a small  group  of  students  to  serve  as  technology  tutors  (named  by  students  as  the  Midland 
Computer  Masters  or  "MCMs").  This  group  of  18  students  met  one  hour/week  to  receive  specialized  training  on 
computer  hardware  and  software,  The  purpose  of  this  ongoing  project  is  threefold:  I)  to  develop  a successful 
technology  program  in  an  elementary  school  setting,  2)  to  target  students  as  technology  leaders  as  a way  to  increase 
self-esteem,  leadership,  technology,  and  collaborative  skills,  and  3)  to  encourage  the  use  of  available  technologies 
among  teachers  by  providing  readily  available  "experts." 

The  current  study  represents  the  second  in  a series  of  year  long  studies  that  have  taken  place  at  the  school, 
investigating  the  effectiveness  of  our  integration  efforts.  Results  from  the  first  year  were  used  to  inform  planning, 
implementation,  and  evaluation  strategics  used  in  the  current  study.  The  primary  research  question  that  guided  our 
study  was,  "What  impact  does  specialized  student  training  have  on  teachers , students,  and  the  school?" 
Specifically,  we  asked: 

• What  changes  occur  in  teachers'  uses  of  technology  as  well  as  their  reasons  for  use? 

• What  changes  occur  in  students'  confidence  with  technology,  basic  computer  skills,  and  self-esteem? 

• What  changes  occur  in  the  school's  technology  culture,  including  use  of  student  trainers? 

Methods 

We  used  qualitative  methods  to  examine  changes  in  the  teachers,'  students,'  and  school's  use  of  technology. 
We  examined  teachers*  and  students'  uses  at  the  beginning  of  the  year,  prior  to  implementing  our  after-school 
training  program,  then  examined  technology  use  at  the  end  of  the  year  to  document  the  types  of  changes  (if  any)  that 
had  occurred.  In  addition,  teachers'  and  students'  attitudes  toward  technology  use  were  assessed,  as  well  as  their 
perceptions  of  the  changes  that  had  occurred  in  themselves,  their  studcnts/lcachcrs,  and/or  the  school.  These 
perceptions  were  then  compared  to  the  researchers’  perceptions  of  change. 

Role  of  the  Researchers 

The  researchers  involved  in  this  study  included  1 faculty  member,  5 graduate,  and  2 undergraduate  students 
from  the  local  university.  Three  of  the  graduate  students  had  been  with  the  project  since  its  inception;  the  faculty 
member  and  undergraduates  joined  at  the  start  of  the  second  year.  Five  Midland  teachers  and  the  librarian  served  as 
"co-chairs"  during  the  second  year,  jointly  overseeing,  with  a university  team  member,  various  components  of  the 
project  (planning,  implementation,  funding,  and  research).  Co-chairs  met  monthly  to  plan  training  and 
implementation  strategies  and  to  determine  how  to  solve  logistical  and  other  problems. 

Description  of  Site  and  Participants 

Midland  Elementary,  a local  Professional  Development  School  (PDS),  is  one  of  1 1 elementary  schools  in 
the  corporation  with  a total  enrollment  of  281  students,  representing  approximately  201  families.  The  student  body 
consists  of  a mixture  of  lower  and  middle  class  students,  including  31  students  with  special  needs.  The  participants 
in  the  study  included  all  13  teachers  (2  teachers  each  in  grades  K-5  with  an  additional  teacher  in  a multi-age  primary 
classroom),  18  students  selected  as  tutors,  the  librarian,  and. the  principal.  Table  1 summarizes  demographic 
information  for  Midland  staff  participants  (using  pseudonyms). 

At  the  time  of  the  study,  Midland's  technology  resources  were  limited.  Each  teacher  had  one  Power  Mac  in 
his/her  classroom.  To  facilitate  student  training  during  Tech  Time  (weekly  1/2  hour  sessions  for  all  students  grades 
1-5)  and  MCMs  (after  school  training  for  18  technology  tutors),  teachers  had  to  roll  their  computers,  on  carts,  to  the 
library  on  Wednesday  afternoons  so  they  would  be  available  for  Tech  Time  on  Thursdays  and  Fridays.  Thus, 
computers  were  in  the  classrooms  three  days  a week  and  in  the  library  the  other  two. 
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Table  I.  Demographics  of  Midland  Staff  Members . 


Participant 

Role/Grade 

Years  of  Experience 

Bea 

Teach  cr/K 

10 

Chris 

Teacher/ 1 

4 

Ethel 

Teach  er/3 

40 

Janet 

Teacher/K 

8 

Janette 

Tcacher/MAP  (1-2) 

24 

Julie 

Tcacher/2 

5 

Lana 

Teacher/M  AP  (1-2) 

30 

Marlene 

Tcac  her/5 

30 

Micelle 

Tcachcr/MAI  (3-5) 

17 

Molly 

Tcachcr/MAlb  (3-5) 

25 

Patty 

Tcacher/M  APa  0-2) 

26 

Polly 

Teacher/MAI  (3-5) 

14 

Stephan 

Tcachcr/4 

23 

Average  =18  years 

Nancy 

Media  Specialist 

5 

Deborah 

Principal 

10 

Note.  aMuIti-agc  Primary.  ^Multi-age  Intermediate 


Procedures 

In  the  fall  of  1996,  Midland  classroom  teachers  in  grades  3-5  were  asked  to  refer  students  for  the  after- 
school  technology  program  based  on  students'  current  computer  skills  (both  high  and  low  levels  were  included), 
ability  to  quickly  learn  computer  skills,  and  ability  to  help  others  learn  computer  skills.  Initially,  18  students  were 
invited  to  participate.  This  included  5 third,  6 fourth,  and  7 fifth  graders  and  consisted  of  10  girls  and  8 boys.  Ten  of 
the  students  met  in  the  school's  library  for  an  hour  each  week  on  Thursdays;  the  other  8 students  met  on  Tuesdays  in 
the  central  administration's  Mac  lab.  Three  second  graders  (a  girl  and  2 boys)  , lined  the  Tuesday  group  in  the 
spring.  Students  learned  about  desktop  management,  word  processing  and  drawing  applications,  HyperStudio  stack 
development,  and  the  Internet. 

Data  Collection 

To  investigate  changes  that  occurred  over  the  year,  survey  and  interview  data  were  collected  from  both 
student  trainers  and  teachers  in  the  fall  and  the  spring. 

Data  sources — students 

Harter’s  Self-Perception  Profile  (Harter,  1984)  was  administered  to  assess  changes  in  student  trainers’  self- 
esteem. The  20  student  trainers  were  interviewed  to  assess  changing  attitudes  toward  computer  technology  as  well 
as  toward  the  after-school  program.  Technology  projects  (c.g.,  word  processed  stories,  HyperStudio  stacks) 
indicated  changes  in  technology  skills  and  served  as  triangulation  data. 

Data  sources — teachers 

All  13  teachers  completed  an  open-ended  survey  in  the  fall  to  assess  current  uses  of,  and  attitudes  toward, 
computer  technology.  Seven  teachers  were  interviewed  to  gather  additional  data.  Follow-up  surveys  were  gathered 
and  interviews  conducted  with  12  teachers,  as  well  as  the  principal  and  media  specialist,  in  the  spring  (the  one  non- 
responding  teacher  was  retiring  at  the  end  of  the  year  and  was  a non-computer  user).  Open-ended  statements 
describing  current  technology  use  (referred  to  as  "workshop  reflections")  were  also  collected  at  two  technology  in- 
services held  during  the  year. 

Data  Analysis 

The  university  research  team  worked  collectively  to  analyze  the  data.  Initially,  each  researcher  took 
responsibility  for  analyzing  one  set  of  data,  (c.g.,  student  surveys),  employing  a constant  comparative  method  of 
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analysis  lo  identify  tentative  patterns  in  the  data.  After  sharing  early  codings  and  themes,  we  exchanged  data  sets 
and  performed  similar  analyses  on  a team  member's  data  set.  Regular  meetings  were  held  throughout  the  year  lo 
compare  and  synthesize  emerging  findings  across  data  sources. 


Interpretation  of  Results 

Guided  by  the  overarching  research  question,  "What  impact  docs  specialized  training  have  on  teachers, 
students,  and  the  school?"  we  examined  changes  that  occurred  in  classroom  teachers  (amount  and  types  of  uses  of 
technology,  reasons  for  use),  student  trainers  (confidence  with  technology,  technology  skills,  and  self-esteem),  and 
the  school  (technology  culture,  use  of  student  trainers).  Data  were  collected  from  five  sources  (teacher  and  student 
interviews  and  surveys,  and  teacher  workshop  reflections)  in  the  fall  and  spring  so  that  changes  in  use  could  he 
assessed  across  lime.  We  examined  changes  in  teachers  and  students  first,  then  drew  on  this  information  lo  describe 
changes  that  occurred  at  the  school  level. 

Teacher  Changes 

Teachers'  use  of  technology  was  reviewed  both  in  terms  of  how  (the  purpose  lo  which  it  was  directed)  and  why 
technology  was  used  (the  supporting  rationale). 

Teacher  Use-Z/ow  Technology  Was  Used 

Teacher  use  was  classified  into  two  main  types:  professional  and  instructional.  Professional  uses  included 
teachers'  use  of  the  computer  to  communicate  with  parents  through  newsletters,  to  create  instructional  materials,  and 
lo  locate  and/or  organize  professional  or  instructional  information.  Instructional  uses  of  the  computer  included  any 
use  in  which  students  interacted  with  computer  programs,  cither  directly  in  the  form  of  some  type  of  CAI  program, 
instructional  game,  or  tool  application  (c.g.,  word  processing,  HyperStudio),  or  indirectly  in  which  the  teacher  used 
a CD-ROM  or  laserdisc  to  present  information  to  the  students. 

Professional  uses  of  technology.  In  the  fall,  nine  of  the  13  teachers  mentioned  that  they  used  the  computer 
primarily  for  writing  newsletters  and  creating  instructional  materials.  Locating  and  organizing  information  via 
databases  (n=l ),  CD-ROM  resources  (n=l ),  and  by  creating  tables  in  Word  (n=  5)  were  noted  less  frequently. 

In  the  spring,  these  same,  as  well  as  additional,  uses  of  the  computer  were  reported  by  more  teachers. 
Twelve  teachers  reported  using  the  computer  to  write  newsletters;  12  used  it  to  create  instructional  materials;  4 
reported  using  databases,  and  6 used  CD-ROM  information  resources.  Additionally,  II  teachers  were  using  the 
computer  for  some  type  of  record  keeping  and  3 teachers  noted  the  use  of  e-mail.  Thus,  more  teachers  were  making 
use  of  more  applications  than  noted  in  the  fall.  As  Nancy,  the  media  specialist  pointed  out,  " The  staff  is  using  the 
computer  much  more.  And  more  members  of  the  staff." 

However,  it  is  important  to  remember  that  this  data  indicates  the  types  of  uses  teachers  made  of  the 
computer  and  not  the  amount  of  use.  Survey  data  indicates  that  teachers'  use  increased  slightly  from  an  average  of 
2.4  hours  a week  lo  approximately  5.5  hours  a week.  In  addition,  teachers'  ratings  of  comfort  levels  with  computer 
applications,  on  a scale  from  I (not  comfortable)  to  4 (very  comfortable),  increased  for  word  processing  (from  3.3  lo 
3.9),  yet  showed  little  change  for  databases,  spreadsheets,  or  graphics. 

Instructional  uses  of  technology.  In  the  fall,  ten  of  the  13  teachers  mentioned  some  type  of  instructional 
use  of  the  computer.  Classroom  uses  revolved  primarily  around  the  use  of  prepackaged  CAI  programs  and 
computer  games  (n=7),  and  these  uses  occurred  most  frequently  within  the  context  of  learning  centers  (referred  lo 
here  as  stations).  Five  teachers  mentioned  using  CD-ROMS  or  laserdiscs  in  instruction,  3 teachers  noted  students' 
use  of  word  processing,  and  3 teachers  noted  students'  use  of  HyperStudio. 

In  the  spring,  there  was  only  one  teacher  who  did  not  mention  using  the  computer  for  instructional 
purposes.  Ten  teachers  mentioned  using  various  CAI  programs,  nine  teachers  were  noted  as  using  instructional 
games,  and  six  teachers  were  noted  as  using  CD-ROM  programs.  Additionally,  11  teachers  mentioned  that  their 
students  were  using  word  processing  for  creative  writing,  and  seven  teachers  noted  the  use  of  HyperStudio  for 
projects  and  reports.  Six  teachers  noted  that  their  students  were  using  graphics  applications  to  enhance  their  reports. 
A couple  of  teachers  also  noted  that  their  students  were  occasionally  using  other  applications,  specifically  the 
Internet,  databases,  and  spreadsheets. 

In  comparison  to  the  fall,  instructional  uses  were  more  varied  (additional  applications  were  mentioned)  and 
more  common  (more  teachers  were  using  more  applications)  in  the  spring.  However  the  primary  context  for  use 
continued  to  be  that  of  a learning  station.  This  finding  from  teacher  interviews  was  supported  by  the  principal's 
observation.  When  asked  to  describe  how  her  staff  was  using  technology,  the  principal  described  two  primary  types 
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of  use:  In  a loi  of  classrooms,  kind  of  the  classic  example  of  "if  you  get  your  work  done  you  can  have  computer 
time."  Others  will  use  it  as  a station  during  discovery  or  center  time. 

Thus  our  results  suggest  that  by  the  end  of  the  1997-98  school  year  the  teachers  had  changed,  to  some 
extent,  in  terms  of  what  they  used,  but  not  in  terms  of  how  that  use  was  structured.  Learning  stations,  or 
independent  remedial  or  enrichment  work,  appeared  to  be  the  main  types  of  uses  of  technology.  These  uses  seemed 
to  fit  relatively  easily  into  teachers'  current  practices  and  seemed  to  support  the  teachers'  ideas  about  the  type  of 
instruction  needed  to  reach  their  students.  This  finding  is  similar  to  that  reported  in  the  literature:  teachers'  initial 
uses  of  computers  tend  to  be  directed  toward  supporting  traditional  goals  (Hadley  & Shcingold,  1993; 
Marcinkicwicz,  1993).  Becker  (1991)  suggested  that  this  is  true  even  for  teachers  with  an  abundance  of  equipment. 
Teachers  favored  traditional  programs  of  instruction  that  were  "not  very  different  than  what  would  be  followed 
without  the  computer"  (p.  8). 

Teacher  Use~W/zy  Technology  Was  Used 

Reasons  for  professional  use.  Teachers  used  technology  because  they  perceived  that  it  increased  their 
efficiency,  enhanced  their  professional  image,  and/or  increased  the  appeal  or  effectiveness  of  their  instruction.  The 
following  quotes  exemplify  these  perceptions: 

• efficiency.  A computer  is  so  much  easier  (than  hand  typing)~you  type  it,  file  it,  and  then  pull  it  up  later. 
You  don't  have  to  take  so  much  time  (Lana,  fall). 

• professional  image:  1 like  to  do  books  for  the  kids,  and  the  print  is  so  much  better  than  my  handwriting.  I 
think  it  makes  things  look  neater,  sharper,  and  more  professional  (Dea,  spring). 

• appeal : I create  a lot  of  my  own  materials— if  I'm  going  to  make  up  math  story  problems  I'll  use  the  kids 
names  ...  so  I can  make  things  closer  to  their  interests  (Julie,  fall). 

• effectiveness : We  can  pick  a theme,  or  a topic  in  science,  and  wc'U  find  pictures  and  short  videos  on  the 
laserdisc  to  support  the  ideas,  which  is  wonderful  (Molly,  spring). 

Reasons  for  instructional  use.  We  analyzed  the  reasons  teachers  gave  for  using  technology  in  instruction 
and  identified  five  main  reasons  for  use:  1)  to  reward  good  behavior  or  effort,  2)  to  build  and/or  reinforce  skills,  3) 
to  enrich  or  enhance  the  current  curriculum,  4)  to  expand  (add  more  depth  or  related  study  to)  the  curriculum  and  5) 
to  change  or  transform  the  curriculum.  Although  none  of  the  teachers  actually  described  using  technology  to  change 
the  curriculum,  this  category  was  included  based  on  the  literature,  as  well  as  on  our  observation  that  some  teachers 
seemed  to  be  moving  in  this  direction.  The  majority  of  uses  that  teachers  described  centered  around  the  first  three 
reasons.  Each  reason  for  use  is  exemplified  below  with  a quote  from  one  of  the  teachers: 

• Reward:  It's  a good  motivator  to  say,  "I'll  make  sure  you  get  to  be  first  on  the  computer  if  you  will  just  do 
this."  I have  one  child  who  it  works  as  a real  good  incentive  for  him  because  he  really  just  flat  out  wants  to 
refuse  to  work  and  so  I can  use  the  computer  as  an  incentive  (Julie,  fall). 

• Reinforce:  Using  tht  computer  to  expand  on  ideas  helps  kids  understand  ana  retain  information,  helps 
kids  review  skills  (Patty,  spring). 

• Enrich:  Technology  integration  means  supporting  my  teaching  and  ideas  and  the  program  that  I'm  in.  It’s 
nice  to  be  able  to  (use)  the  CD-ROMs  and  things  that  are  already  prepared.  Some  of  them  do  a nice  job  of 
integrating  ideas.  It's  nice  sometimes  to  have  something  right  on  hand  that  you  don’t  have  to  spend  time 
developing  (Michelle,  spring). 

• Expand:  Technology  integration  means  that  all  (students)  can  use  the  computer  in  a way  where  the 
technology,  whatever  it  may  be,  in  a way  that  helps  foster  a better  learning  experience  or  a more  creative 
xperiencc  (Nancy,  spring). 

• Change:  Technology  integration  means  using  technology  to  develop  the  curriculum,  to  continually  create 
new  curriculum  (Marlene,  fall,  1997). 

Teachers  at  Midland  typically  did  not  describe  using  technology  to  provide  pari  of  the  "core"  curriculum 
but  rather  to  provide  additional  or  more  motivating  ways  for  students  to  learn  skills  or  topics  that  had  already  been 
introduced.  Figure  I summarizes  our  interpretation  of  each  teacher's  primary  reason(s)  for  use  based  on  interview 
and  survey  data,  and  also  helps  to  illustrate  where  the  school  was,  as  a whole,  in  terms  of  use.  Wc  observed  little 
change  in  teachers'  reasons  for  use  from  fall  to  spring.  Given  that  technology  use  is  likely  to  represent,  rather  than 
shape,  beliefs  about  teaching  and  learning,  major  change  was  not  expected. 
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Figure  !.  Teac  hers  ' primary  reasons  for  using  technology  tn  the  classroom. 
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We  think  ol  these  five  types  ol  uses  (e.g.,  reward,  remloree.  enrich,  expand. 

change)  as  being  on 

continuum  with  higher  levels  of  integration  on  ihc  right.  Whereas  a "reward"  type  of  use  can  occur  complete!) 
outside  the  eumeulum.  a "clange"  type  o\  use  leads  to  an  evolving  airriculum--onc  liiat  is  continually  heuiji 
invented  as  new  opportunities  and  possibilities  arise  (Hooper  A Rcihcr,  1 9C>5 ) . In  tins  studs,  most  teacheis 
emphasized  onl>  one  i \ pc  of  use.  although  one  teacher  (baits)  frequently  mentioned  both  mops atioiul  aiul 
enrichment  uses,  as  noted  In  the  extra  X in  the  figure. 

Summary 

based  sole!)  on  the  number  of  teachers  who  used  the  computer,  and  the  number  ol  ss.tss  dies  leporled 
using  it.  our  results  suggest  dial  there  ssas  an  increase  in  technolngs  use  by  teachers,  loi  both  professional  and 
instructional  purposes,  from  fall  to  spring  Vet  these  increases  in  use  did  not  appear  to  he  accompanied  in  an\ 
related  shifts  in  instructional  approaches  or  processes,  batty’s  quote  illustrates  how  most  teachers  used  technology  to 
support  current  curricula: 

I like  to  see  the  technology  brought  m and  related  to  what  I'm  doing.  And  I think  n takes  a well  planned 
teacher  to  do  that  I think  1 need  to  plan  for  lunv  could  this  computer  help  me  expand  on  this  idea  or  how 
could  the  computer  help  gel  this  scientific  term  ingrained  in  their  little  minds?  I low  can  lech  help  res  tew 
this  math  skill  that  I have  alrcads  taught  them? 

In  general,  the  changes  that  teachers  noted  in  themselves  related  to  increases  in  comfort,  confidence,  and 
interest  rather  than  to  changes  in  leaching  methods  or  classroom  organization.  As  Molly  explained  in  the  fall:  "As 
fai  as  classroom  processes,  (they)  haven't  changed  at  all"  and  again  in  the  spring.  'Technology  hasn’t  changed  the 
w ay  1 leach.  I look  at  technology  as  a support  to  the  way  I leach." 

Student  Changes 

Differences  between  students'  pre  and  post  Self-Perception  Profiles  were  not  statistically  significant  u - 
.029;  p>  .05),  vet  survey  and  interview  data  from  both  teachers  and  students  provided  evidence  of  change  in 
students'  confidence,  technology  skills,  and  self-esteem.  Comments  were  classified  as  related  to  confidence  if  they 
rclcrred  to  students’  comlort  with  technology,  with  helping  others  with  technology,  aiul/or  with  a growing 
conviction  of  then*  technology  skills.  Comments  were  classified  as  related  to  increased  technology  skill*  iflho\ 
referred  to  new  tasks  that  students  could  complete  on  the  computer.  Comments  were  classified  as  related  to  self- 
esteem if  they  referred  to  children's  increased  sense  of  self- worth  or  pride  in  their  accomplishments. 
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Increased  Confidence 

Students  made  many  comments  that  reflected  their  growing  confidence  with  technology.  When  asked  the 
question  "I  used  to  (do  something  in  regard  to  technology),  hut  now  I (do  something  else  concerning  technology)," 
nearly  all  of  the  students  answered  in  a way  that  suggested  an  increase  in  both  skills  and  confidence.  For  example, 
Anna  remarked: 

I used  to  not  go  on  computers  and  I used  to  think  that  computers  were  for  the  brainy  kids  and  they  weren’t 

for  me.  And  they  were  loo  hard  for  me.  But  now  I know  that  computers  can  be  fun  sometimes  and  they 

aren't  always  hard  and  they  aren't  for  brainy  kids,  they  can  also  be  for  me  loo. 

Ashley  expressed  a similar  idea: 

I used  to  be  like,  "It's  never  gonna  happen,  I'm  never  gonna  understand  technology.  Computers?  Too  hard." 

But  now  I'm  like,  “Computers?  Easy.  HyperStudio,  Claris  Works.. .They're  just  like  the  basics."  They  used 

to  be  really  hard  for  me,  but  now  they're  really  easy. 

Students  were  generally  not  afraid  to  help  when  their  teachers  experienced  difficulties  with  the  computer, 
even  if  they  were  not  certain  how  to  fix  the  problem  themselves.  Tony,  a third  grader  said,  "...she  goes  'Docs 
anybody  know  how  to  turn  the  computer  off?'  and  I go  Til  try  to.’  So  I tried  it  and  it  worked."  Anna  commented, 
"Sometimes  the  teachers  get  stuck  on  the  computer  and  I'm  like  'Well,  maybe  if  you  take  it  this  way,  it'll  be  able  to 
work.'  And  then  they  try  it  and  sometimes  it  works  and  sometimes  it  doesn't." 

Students  also  liked  being  able  to  help  the  other  students  in  their  classrooms.  Comments  such  as  Erin's:  "I 
like  doing  it  (the  MCM  program)  because  I gel  to  learn  new  things  and  then  I can  show  it  to  other  kids,"  and 
Danielle's:  "...I'll  help  them.  Sometimes  I'll  teach  them  new  things  like  on  databases  and  spreadsheets.  I'll  teach  a 
little  bit  on  the  Quick  Take  camera.  Just  simple  things  like  that"  both  illustrate  the  confidence  students  displayed 
when  helping  others. 

Teachers  also  noted  how  students'  confidence  had  grown.  As  Janette  pointed  out,  "The  children  arc 
fearless.  They  arc  not  afraid  to  try  things  and  they  genuinely  enjoy  it  (technology)."  Stephan  noted  how  "several 
students  helped  other  students  with  computer  skills."  The  principal  indicated  that  she  had  observed  increases  in 
"students' comfort  level  in  knowing  some  of  the  different  software  packages  and  how  to  use  them." 


Increased  Computer  Skills 

Most  likely,  the  increases  in  students'  confidence  were  related  to  their  growing  skill  levels.  Students  were 
certain  that  they  learned  a great  deal  as  MCMs.  When  asked  at  the  end  of  the  year  what  they  had  learned,  students 
made  comments  such  as,  "Everything.  I never  knew  anything  about  a computer"  (Shanca).  Jason  described  a few  of 
the  specific  skills  he  had  learned,  "I've  learned  lots  of  things  like  you  can  get  into  writing  Claris  Works,  you  can  get 
a picture  from  up  there,  cut  it,  then  go  back  into  HyperStudio  and  if  they  don't  have  that  same  picture  you  can  paste 
it  on  there  and  then  you  have  a picture  from  Claris  Works.  I've  just  learned  a lot  of  things."  Laura  also  listed  some 
of  the  specific  skills  she  had  learned:  "...how  to  do  some  stuff  like  with  the  ZapShot  camera  and  load  it  on  to  the 
computer.  And  we're  starting  to  use  the  Internet  and  how  to  find  things  on  the  Internet."  Similar  comments  were 
made  by  nearly  all  students  interviewed. 

Teachers  also  commented  on  the  growing  skill  level  of  the  students.  Nancy  explained,  " We  arc  seeing  a lot 
more  usage  of  the  computers.  As  a whole  the  students  arc  learning,  understanding  it  better,  the  students  know 
(more),  than  for  the  most  part,  the  staff  members." 

Increased  Self-Esteem 

Although  students  did  not  make  any  direct  comments  about  increases  in  sclf-cstceni,  we  can  infer  this  from 
the  way  they  described  their  growing  skills  and  confidence.  When  Anna  stated  how  "technology  is  not  just  for 
brainy  kids  but  for  me  too"  she  conveyed  pride  in  what  she  had  accomplished.  When  Ashley  described  how  she 
"caught  on  pretty  quickly"  and  how  she  "just  gets  it  right"  when  she  guesses  what  to  do,  she  also  expressed  a sense 


of  pride.  Nick  described  how  lie  felt  special  because  of  his  new  skills:  "I  used  to  only  do  regular  stuff  on  the 
computer  that  everybody  knows  but  now  I do  more  higher  level  technology  stuff  because  I know  more." 

Teachers  made  a number  of  comments  about  how  students,  particularly  their  "at  risk"  or  "difficult" 
students,  had  excelled  in  the  technology  program  and  had  experienced  increased  self-esteem.  For  example,  Marlene 
noted: 


Kids  can  be  successful  even  if  they  have  not  been  successful  with  the  regular  curriculum.  (Technology) 
improves  the  self-concepts  of  all  students,  especially  at  risk  students.  . . I think  we  should  continue  the 
Tech  program.  The  children  enjoy  it,  they  feel  proud,  they  feel  successful.  One  of  my  students  goes  to 
another  classroom  and  helps  another  teacher,  which  makes  him  (work)  better. 

Molly  described  it  this  way: 

I've  found  that  many  of  our  kids  who  have  a handicap  are  very  successful  on  computers  and  it  makes  them 
very  confident,  happy,  and  proud.  It’s  just  really  fun  to  watch  them.  In  many  eases  those  kids  turn  around 
and  become  helpers  to  students  that  you  would  assume  would  catch  on  immediately. 


School  Changes 

The  principal  and  the  teachers  perceived  that  the  school,  as  a whole,  had  made  some  important  changes  in 
technology  use.  At  the  start  of  the  study  (fall  of  1996),  teachers  were  asked  to  describe  the  role  that  technology  had 
played  at  Midland  in  the  past.  Many  teachers  described  the  limited  role  that  technology  had  played  to  date.  Others 
elaborated,  indicating  that  the  primary  use  of  the  computer  had  been  in  the  form  of  drill  and  practice  programs, 
instructional  games,  and  "a  little"  word  processing.  In  the  spring,  teachers  made  the  following  comparisons  to 
describe  their  perceptions  of  how  technology's  role  had  changed: 

Patty:  We  used  to  use  computers  sporadically  but  now  we  have  a plan. 

Julie:  We  used  to  have  limited  use,  now  we  integrate  with  instruction. 

Lana:  We  used  to  (just)  talk  about  it,  but  now  we  use  it. 

Dca:  We  used  to  let  the  computer  sit;  now  we  have  instructional  programs  on  it  for  the  kids. 

Marlene:  We  used  to  play  games,  now  we  enrich  all  areas  of  the  curriculum. 

Polly:  We  used  to  view  computers  as  reward  for  finishing  work,  now  sec  them  as  more  of  a tool  to  assist 

students. 

Deborah:  "We’ve  probably  grown  3, 4,  5 years  this  year,  in  comparison  to  past  years." 

These  changes  that  the  teachers  and  principal  noted  (in  planning,  frequency  of  use,  and  reasons  for  use) 
were  perceived  as  contributing  to  a climate  of  change  at  Midland,  with  a focus  on  technology.  As  Marlene  noted,  "I 
think  it  (our  technology  focus)  has  helped  to  bring  the  staff  together  and  we  feel  a commonality."  The  principal 
staled  that  the  changes/growth  at  the  school  were  due,  not  to  specific  factors,  but  to  this  change  in  culture,  "It’s  not 
just  the  MCMs  and  Tech  Friday.  I think  it’s  an  entire  package,  a focus,  a concerted  effort,  and  also  a breaking  down 
of  barriers.  . . . I'm  not  sure  what  the  catalyst  was,  but  I think  it's  probably  a brew  that  mixes." 

As  a whole,  teachers  in  the  school  seemed  to  be  making  a conscious  effort  to  include  the  computer  in  their 
curricula  in  whatever  ways  fit  their  teaching  styles  best-whether  as  a remedial  or  reward  tool,  or  as  part  of  an 
instructional  center.  Talking  about,  and  working  toward  using  the  computer,  had  become  a natural  part  oi  the  school 
culture.  This  changing  culture  was  also  reflected  in  teachers'  altitudes  toward,  and  increased  uses  of,  the  MCMs  as 
described  in  the  next  section. 

Attitudes  Toward  and  Use  of  Student  Trainers  (MCMs) 

Teachers'  responses  to  the  MCM  program  were  positive  throughout  the  year.  In  the  initial  interview  Patty 
commented,  “I  think  the  Tech  program  has  had  the  biggest  influence  on  our  children’s  ability  to  use  the  computers.” 
In  the  spring  Michelle  said,  “I  think  what  we’re  doing  is  really  good.  I’m  hoping  it  will  be  something  that  over  the 
next  few  years  will  really  grow.” 

As  the  year  progressed,  teacher  use  of  the  MCMs  increased.  At  the  beginning  of  the  year,  teachers  reported 
no  use  of  the  student  helpers.  By  the  end  of  the  year,  teachers  indicated  that  the  MCMs  were  being  used  by  both 


students  and  teachers.  Janet  reported,  “I  have  an  older  student  [a  fifth  grader]  working  with  one  of  my 
kindergartners,  leaching  her  HyperStudio.”  Julie,  the  2nd  grade  teacher  commented,  "We  used  to  rely  on  teachers  to 
he  experts,  but  now  the  MCMs  are  knowledgeable."  One  teacher  described  how  she,  herself,  had  used  a student 
trainer: 


We  had  to  bring  (to  the  second  inscrvicc)  something  that  we  had  done  using  it  (HyperStudio).  I hadn’t 
done  anything  on  it.  I thought  "Oh  great,  what  am  I going  to  do?"  So  I grabbed  a 5th  grader  who  is 
excellent  in  it.  She  was  wonderful.  1 was  absolutely  amazed  at  her  knowledge  on  it.  I learned  a lot  from 
her  and  I think  that’s  good  that  the  kids  see  that  we  don’t  know  everything  and  that  must  make  them  feel 
kind  of  good  that  they  can  help  us  out  too.  (ESL  Teacher) 

Teachers'  positive  attitudes  toward  the  program  may  also  have  been  enhanced  as  they  began  to  see 
additional  benefits  of  the  Tech  program.  Student  enthusiasm  and  computer  expertise  increased.  Students  also  began 
solving  technology  related  problems. 


Perceptions  of  the  Students 

Based  on  teachers'  comments,  ^ ’ "Oogy  use  at  Midland  had  changed  substantially  during  the  year. 
However,  teachers  tended  to  perceive  gre  . - (by  both  teachers  and  students)  than  students  did.  For  example, 
one  teacher  staled,  "My  class  uses  compu  rs,  J players,  laser  disc  players,  and  ZapShot  cameras  throughout  the 
jculum,"  yet  one  of  her  students  said:  "The  kids  don't  use  that  much  technology  anymore,  other  than  games.  For 
typing  reports  and  stuff  like  that  we  don’t  really  use  the  classroom  computer.  It  just  sits  there  most  of  the  lime." 
Similar  comments  were  made  by  other  students: 

We've  used  (computers)  during  my  free  lime-which  is  hardly  ever  (Ashley). 

I don't  ever  use  it  for  my  school  work.  Once  we  did,  but  we  don't  do  it  anymore.  We  used  to  play  on  the 

computer  at  recess  in  the  winter  (Shanca). 

It's  not  clear  why  students  perceived  different  levels  of  use  than  the  teachers.  Maybe  because  students  were 
so  anxious  to  use  the  computers  they  thought  that  they  never  got  to  use  them  enough.  Or  maybe  the  teachers' 
percept  ions/definitions  of  use  were  broader  than  the  students.  Whatever  the  reason,  we  realize  that,  in  the  future,  we 
need  to  expand  our  data  collection  methods  to  include  classroom  observations.  These  will  enable  us  to  verify  how 
often  students  and  teachers  arc  using  the  computers  as  well  as  how  they  are  being  used. 


Perceptions  of  the  Researchers 

Teachers  at  Midland  defined  technology  integration  differently  than  the  researchers.  Whereas  our  definition 
was  directed  more  toward  "changing"  the  curriculum,  teachers  in  this  study  typically  defined  technology  integration 
as  using  technology  "to  present,  reinforce,  or  motivate  children  to  interact  with  the  (current)  curriculum."  For 
example,  a teacher  who  stated  that  she  used  technology  to  "enrich"  the  curriculum,  elaborated  by  explaining  that 
technology  provided  an  additional  way  for  her  to  present  the  curriculum  to  the  students.  Another  teacher  defined 
technology  integration  as  "using  technology  to  enhance  what  you  arc  teaching;  helping  students  learn  what  I need  to 
teach  them."  In  these  eases,  enrich  meant  providing  students  with  more  of  the  same  curriculum  as  opposed  to  taking 
the  students  beyond  what  was  currently  prescribed  to  explore  new  topics  or  old  topics  in  more  depth  and  with 
greater  involvement.  "I'm  not  dependent  upon  the  computer  to  teach  them  something  but  more  (to)  reinforce  things 
that  I've  taught." 

This  approach  to  technology  integration  is  consistent  with  how  other  teachers,  in  the  literature,  have 
described  "getting  started"  with  technology.  Dwyer  (1996)  reported  that  when  teachers  started  using  technology  in 
the  ACOT  classrooms  there  was,  initially,  very  little  change  in  classroom  practice.  However,  as  teachers  gained 
confidence  and  skill  with  the  technology,  and  as  they  witnessed  changes  in  their  students'  motivation,  performances, 
and  interactions  with  each  other,  a shift  occurred: 

Teachers'  own  growing  prowess,  coupled  with  observable  changes  in  their  students'  work  and  forms  of 

interaction,  opened  the  staff  to  the  possibilities  of  redefining  how  they  went  about  providing  opportunities 


for  students  to  learn.  Technology  was  acknowledged  as  the  catalyst  for  new  perspectives  and  practices  (p. 

24). 

Midland  School  has  only  recently  embarked  on  the  technology  change  process.  The  teachers'  current 
approaches  to  use  fit  comfortably  within  their  current  levels  of  knowledge,  skill,  and  confidence.  However,  as  these 
continue  to  increase,  and  as  students'  confidence  and  skill  also  continue  to  grow,  change  may  occur  more  rapidly. 

Discussion 

Our  work  with  Midland  School  provides  a mini-case  study  of  the  educational  change  process.  Educational 
change  has  been  noted  as  being  a particularly  slow  process,  especially  when  it  conies  to  adopting  new  technologies 
(Cuban,  1986;  1990).  The  literature  suggests  that  it  takes  3-5  years  for  teachers  to  become  "integrated"  technology 
users  (Hadley  & Sheingold,  1993).  Even  with  the  extra  effort  and  time  contributed  by  Project  START,  wc  arc  not 
sure  that  we  have  expedited  this  relatively  long  process.  What  wc  seemed  to  have  accomplished,  however,  was  to 
heighten  teachers’  awareness  of  technology  use,  particularly  for  those  who  had  shied  away  previously.  By  being 
present  in  the  school,  by  constantly  asking  teachers,  students,  and  staff  to  consider  where  they  were  and  where  they 
wanted  to  go  with  technology,  wc  believe  we  helped  increase  the  school's  awareness  of  what  was  happening  (and  to 
some  extent,  what  was  possible)  with  technology  and  thus  contributed  to  the  establishment  of  a technology  culture 
in  the  school. 

Although  Midland  school  did  not  move  as  far  or  as  fast  as  we  had  hoped,  the  principal  and  teachers  all 
perceived  that  major  changes  had  occurred.  The  principal  stated,  "giant  strides  have  been  made."  The  changes  that 
the  teachers  noted  as  having  occurred  at  the  school  level  (sec  p.  15)  arc  cited  in  the  literature  as  being  critical  during 
the  early  stages  of  integration:  having  a plan,  initiating  use,  making  use  relevant,  extending  use  to  new  areas,  and 
reconceptualizing  use  as  a tool  rather  than  a reward  (Hadley  & Sheingold,  1993).  Indeed,  without  these  first  steps, 
more  innovative  and  challenging  changes  are  unlikely  to  occur. 

Midland  teachers  experienced  many  first-order  barriers  to  computer  integration  including  lack  of 
computers,  time,  and  experience,  yet  these  barriers  did  not  prevent  them  from  becoming  "keen  on  technology."  In 
fact.  Midland  provides  a good  example  of  a how  a school  with  few  resources,  but  with  a lot  of  teacher  and  student 
motivation,  can  jump-start  the  integration  process.  During  the  1996-97  school  year,  Midland's  culture  assumed  a 
strong  technology  flavor,  which  wc  attribute  to  many  interrelated  factors:  increased  student  and  teacher  confidence 
and  skill,  use  of  student  trainers,  as  well  as  the  researchers’  presence  in  the  school. 

Although  teachers  were  quick  to  point  out  existing  first-order  barriers,  they  seemed  unaware  of  second- 
order  barriers  operating  in  the  process.  Yet,  in  the  researchers'  view,  teachers'  existing  instructional  practices  and 
their  beliefs  about  the  role  of  technology  in  the  curriculum  limited  the  extent  to  which  technology  was  used  to 
support  meaningful  learning. 

So  what  happens  now  at  Midland?  Even  though  the  literature  suggests  that  technology  use  can  prompt 
changes  in  teaching  practices  and  beliefs  that  might  not  occur  otherwise  (Goldberg  & Richards,  1995;  OTA,  1995), 
we  saw  little  evidence  of  these  types  of  changes  occurring  at  Midland.  Teachers  were  not  only  content  with  what 
they  had  achieved,  they  were  proud-they  had  made  important  changes;  they  had  come  a long  way. 

Before  wc  decide  how  best  to  assist  Midland  during  the  next  few  years,  we  need  to  answer  some  difficult 
questions:  Have  Midland  teachers  achieved  an  effective  level  of  technology  integration?  Should  wc  be  content  with 
the  gains  that  have  been  made?  Should/Can  we  challenge  these  teachers  to  search  for  new  uses,  new 
conceptualizations  of,  and  new  visions  for  technology  integration?  (If  so)  How  can  wc  help  teachers  move  beyond 
current  uses,  given  their  existing  teaching  practices  and  expectations  for  technology? 

Although  our  work  with  the  MCMs  has  provided  Midland  with  on-going  technical  support  through  the  use 
of  student  helpers,  what  seems  needed  now  is  a greater  provision  for  instructional  support.  Based  on  the  findings  of 
this  study,  as  well  as  numerous  suggestions  in  the  literature,  we  believe  that  this  support  should  be  in  the  form  of 
helping  teachers  envision  the  kind  of  learning  experiences  they  can  create  for  students  through  technology-based 
learning  activities  and  then  providing  the  resources  (technical,  emotional,  and  instructional)  needed  to  realize  this 
vision.  This  includes  giving  teachers  the  time  and  support  needed  to  observe  exenv’  -ry  models  of  classroom 
technology  use,  to  reflect  on  and  discuss  their  evolving  ideas  with  mentors  and  peers,  an  o collaborate  with  others 
(students  and  teachers)  on  meaningful  tasks  as  they  try  out  their  new  ideas  about  teaching  and  learning  with 
technology. 
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Conclusion 

"Neither  powerful  technology  nor  good  ideas  arc  enough  to  improve  education.  Success  in  using 
computers  in  education  will  come  only  as  a result  of  the  intelligent  and  artful  orchestration  of  many  details  in  the 
classroom"  (Walker,  1996,  p.  102),  the  majority  of  which  arc  determined  by  individual  classroom  teachers. 
Teachers,  not  technology,  arc  the  key  to  whether  technology  will  be  used  appropriately  and  effectively  in 
classrooms.  Thus,  efforts  to  integrate  technology  must  focus  on  teachers,  what  they  believe  comprises  good 
instruction  and  good  learning,  and  how  they  put  these  beliefs  into  practice. 

Rather  than  address  technological  or  pedagogical  needs  separately,  we  propose  that  technology  skills  and 
pedagogical  beliefs  be  developed  simultaneously.  This  might  be  accomplished  by  embedding  technology  training 
within  authentic  activities  that  engage  teachers  in  collaborative  problem-solving  tasks,  relevant  to  their  needs.  In 
this  way  we  can  provide  teachers  with  both  a vision  of  what  "teaching  with  technology"  looks  like,  as  well  as  a 
model  of  the  type  of  learning  experiences  we  arc  asking  them  to  create  for  students. 

Regardless  of  where  teachers  start  in  the  integration  process,  whether  unfamiliar  with  technology,  studcnl- 
ccntcred  pedagogy,  or  both,  teachers  will  need  to  learn  new  skills  related  to  any  or  all  of  the  different  elements 
involved  in  the  integration  process-technology,  instructional  design,  classroom  organization  and  management,  and 
student  assessment,  to  name  a few.  Furthermore,  most  teachers  arc  likely  to  find  themselves  having  to  confront  their 
established  beliefs  about  instruction  and  their  traditional  roles  as  classroom  teachers.  It  is  imperative  that  we  help 
teachers  attend  to  all  aspects  of  technology  integration.  Anything  less,  and  our  efforts  arc  unlikely  to  result  in 
sustained  classroom  implementation  (David,  1996;  Gricst,  1996;  Pea,  1997). 
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Introduction 

Whether  we  like  it  or  not,  the  nature  of  literacy  and  learning  is  being  redefined  by  the  digital  technologies  that  are 
quickly  becoming  a part  of  the  information  age  in  which  we  live. 

(Donald  Leu  Jr.,  1996) 

In  spite  of  some  disagreeing  voices  (Oppenheimer,  1997;  Postman,  1993)  the  advances  that  technology 
offers  to  actual  society  and  education  are  irrefutable  (Freitas,  1997;  Leu,  1996;  Mehlinger,  1995;  Negroponte,  1995). 

In  such  a society,  where  technological,  political  and  social  progress  narrows  more  and  more  the  boundaries 
among  people,  schools  and  countries,  "information  access,  problem  solving,  and  communication  arc  essential  to 
success  in  the  information  age  in  which  we  live".  (Leu,  1997,  p.  63).  Schools  play  an  essential  role  in  preparing 
students  to  work  and  live  in  this  world  (Freitas,  1997;  Magildson,  1997).  Therefore,  teachers  need  to  be  skilled  in  the 
use  and  application  of  these  technologies  in  rich  communicative  contexts:  “[T]hc  role  of  the  teacher  will  be 
essential  in  fostering  an  environment  that  creates  teaching  and  learning  opportunities  and  uses  the  technology  to  its 
best  advantage"  (McHenry,  1997,  p.24). 

The  University  of  Minho,  in  Portugal,  where  we  teach,  has  a great  tradition  in  encouraging  and  supporting 
educational  technology  activities  in  schools  in  a collaborative,  informal  and  practice-centered  way  because  we 
believe,  that  "there  are  few  topics  more  important  than  the  role  of  technology  in  education"  (Hong,  1997,  p.  1 88)  and 
we  believe  also  that  investigators  and  teachers  can  help  each  other.  In  fact,  as  it  was  previously  said  by  one  of  us 
"The  full  sense  of  practice  just  happens  whether  it  has  its  foundations  grounded  in  theory  or  it  provides  a basis  for 
theory  building  " (Freitas,  1997,  p.  15).  In  addition,  to  the  extent  these  technologies  proliferate  schools,  researchers 
are  under  pressure  to  develop  knowledge  useful  both  for  teacher  training  courses,  in  University,  and  for  practitioners 
as  a way  to  avoid  that  "what  had  started  as  a subversive  instrument  of  change  was  neutralized  by  the  system  and 
converted  into  an  instrument  of  consolidation"  (Papert,  1994,  p.39). 

The  fact  is  that  there  is  a growing  body  of  research  focusing  on  educational  technology.  However, 
combining  technologies,  like  Web,  videoconference  and  word  processor,  with  cooperative  work  to  improve 
language  skills,  was  not  done  before,  at  least  in  Portugal.  On  the  other  hand,  the  present  study  differs  from  some 
previous  ones  in  several  ways:  a)  activities  were  carried  out  in  authentic  classroom  context,  rather  than  in  laboratory; 
b)  texts  written  using  the  word  processor  were  based  on  real  purposes  rather  than  on  specially  ones  written  for 
research;  c)  data  collected  include  researchers'  observations,  teachers'  commentaries  and  students’  voices,  until  now 
often  ignored.  The  analysis  was  based  on  grounded  theory  methodology,  one  of  the  most  fruitful  and  used  ways  of 
carrying  out  qualitative  research  when  researchers'  main  objective  is  generating  theory. 

So,  in  this  study,  wc  "does  not  begin  with  a theory,  then  prove  it.  Rather,  one  begins  with  an  area  of  study 
and  what  is  relevant  to  that  area  is  allowed  to  emerge"  (Strauss  & Corbin,  1990,  p.  22). 

Theoretical  guidelines 

Our  approach  (Freitas,  1997),  which  is  not  just  ours  (see,  for  instance,  Wilson,  1996)  is  that  educational 
technologies  must  be  used  in  contexts  promoting  interaction  and  allowing  access  to  various  information  resources 
and  supports.  Through  wide  discussion  with  peers  this  information  is  clarified,  allowing  a greater  understanding  of 
the  subjects  under  study.  This  approach  is  consistent  with  cooperative  learning  and  constructivist  approach 
assumptions. 

Among  the  various  approaches  to  constructivism  (Brooks,  1997;  Brunning,  1995;  Coll,  1996;  Carrctcro, 
1997)  we  adopted  the  Duffy  and  Cunningham  one  as  it  was  referred  by  Ritchie.  They  "identified  two  common 


characteristics  of  constructivism  environments:  a)  students  learn  by  actively  constructing  rather  than  acquiring 
knowledge;  and,  b)  the  purpose  of  instruction  is  to  support  this  construction,  rather  than  the  communication  of 
information"  (1997,  p.27). 

By  and  large,  extensive  research  provided  evidence  that  cooperative  learning  improves  academic 
achievement,  facilitates  both  cross-ethnic  and  cross-sex  inter-group  relationships,  and  increase  self-esteem  (Johnson 
& Johnson,  1981;  Johnson,  Johnson  & Maruyama,  1983;  Slavin,  1985;  Slavin  & Karwait,  1981).  A recent  study 
developed  by  Brush  to  investigate  whether  cooperative  learning  techniques  helped  to  improve  success  with  students 
in  Integrated  Learning  Systems  (ILS)  concluded  that  integrating  cooperative  learning  with  ILS-delivercd  instruction 
is  an  effective  instructional  strategy.  Results  showed  that  students  working  on  math  activities  delivered  by  an  ILS 
performed  better  on  achievement  tests  of  the  content  taught  when  they  completed  the  activities  in  cooperative  pairs 
than  when  they  worked  in  activities  individually  (1997,  p.  61). 

Other  studies  in  this  domain  suggest  that  working  together  and  supporting  each  other  "stimulates  initiative, 
attention  to  details  of  performance,  and  commitment  in  activity  6 more  expert  play  partner  can  facilitate  the  social 
pretense  in  low-play  peers  and  not  at  their  own  expenses"  (Newman,  1997,  p.  17).  Xypas  (1997),  as  others  did  before 
(e.g.,  Smith,  Johnson,  & Johnson,  1981,  1984),  stresses  the  educational  value  of  conflict  that  often  occurs  when 
students  work  together.  He  argues  that  this  conflict  impels  students  to  construct  more  deeply  rooted  arguments  and 
explanations  and  gives  them  the  opportunity  to  assist  one  another.  This  peer  discussion  contributes  to  cognitive  and 
social  gains.  Besides,  such  a supportive  learning  environment  helps  to  reduce  the  negative  statute  of  mistakes,  that 
often  are  a source  of  anxiety  and  stress,  encouraging  students  to  approach  them  as  the  "rolling-carpet"  of  their 
cognitive  and  social  growing  (Astolfi,  1997). 

Creating  learning  situations  that  aim  at  improving  linguistic  skills  and  arc  cognitive  and  emotionally 
stimulating  for  students  is,  probably,  the  greatest  challenge  of  language  learning  teachers.  Current  language  learning 
perspectives,  such  as  communicative  and  whole  language  approaches,  suggest  the  creation  of  authentic 
communicative  situations  in  which  students  a)  use  language  realistically,  as  a real  means  of  communication  in  real 
situations;  b)  learn  and  practice  language  skills  - listening,  speaking,  rcadifng  and  writing  in  a meaningful  and 
integrated  way. 

As  asserted  by  Landsmann  (1995)  language  learning  activities  must  occur  in  contexts  that  promote  active 
learning  since  students  need  opportunities  to  compare  and  share  ideas  as  a means  to  reflect  about  their  oral  and 
writing  productions  and  improve  them. 

The  practice  of  writing  skills  is  also  a privileged  way  to  increase  thinking  skills  (Schneuwly,  1988)  to  the 
extent  that  writing  makes  it  possible  to  reach  the  most  abstract  level  of  language  (Vygotsky,  1979,  1993). 

Recent  approaches  to  writing  suggest  that  it  is  a complex  and  dynamic  process  consisting  of  a set  of  phases 
(d'Aoust,  1989;  Hayes  & Flower,  1980).  These  phases  may  be  focused  or  not,  conscious  or  unconscious  and  they  are 
nor  independent  neither  sequential,  rather  they  interact  in  a recursive  process.  They  include  several  sub-processes, 
organized  and  labeled  in  several  ways  by  different  authors.  Summing  up,  in  the  writing  process  we  have  writing, 
sharing,  revision,  correction  and  evaluation. 

Pre-writing  activities  arc  supposed  to  stimulate  and  shape  ideas;  writing  is  the  phase  in  which  ideas  arc 
materialized  in  the  written  word.  Sharing  allows  the  writer  to  revise  and  improve  his  text  having  in  mind  the 
feedback  given  by  peers  and  teacher.  In  the  revision  and  correction  phases,  students  make  further  improvements: 
words,  sentences,  paragraphs  or  the  whole  text.  This  is  the  very  moment  in  which  the  teacher  can  explore  and 
conduct  educational  work  in  relation  to  language  structure  and  functions.  The  evaluation,  the  final  feedback, 
assuming  usually  the  form  of  assessment,  is  the  final  phase  of  writing 

The  only  ways  to  increase  writing  skills  arc  practicing,  learning  by  doing,  as  suggested  by  constructivist 
approaches.  Writing  tasks  for  several  purposes  and  in  various  curricular  domains  gives  students  these  opportunities 
to  practice  writing. 

Besides,  writing  together  also  gives  students  an  opportunity  to  improve  writing.  By  constant  and 
constructive  feedback  provided  by  peers,  by  reflecting  about  their  own  and  other's  writing,  they  can  increase  writing 
skills.  New  writing  tools,  like  word  processor,  supported  by  teachers  when  necessary,  increases  the  group  work 
potential,  since  they  facilitate  corrections  and  improvements  in  written  texts.  In  addition  such  a supportive 
environment  develops  students  self-confidcncc  in  expressing  themselves,  orally  and  in  writing.  This  is  an  important 
first  step  in  improving  students'  communicative  skills. 

Therefore,  technologies  and  cooperative  learning  may  contribute  to  implement  learning  environments 
based  on  constructivist  principles  that  emphasize  the  central  role  of  communication  in  language  learning. 
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The  Portuguese  Curriculum  framework 

The  last  curriculum  reform  (1989)  prescribes  the  adoption  of  group  work  to  promote  and  improve 
cognitive,  social  and  affective  skills.  Curriculum  guidelines  also  suggest  that:  a)  students  must  have  access  to  new 
technologies  in  various  ways  namely  for  the  searching,  interpreting,  organizing  and  evaluating  information;  b) 
technologies  must  be  envisioned  as  a tool  to  increase  learners  development  of  social  and  cognitive  skills  related  to 
several  curricular  domains.  Portuguese  syllabi  consider  that  reading  and  writing  instruction  must  be  carried  out  in 
several  learning  environments  that  stimulate  interaction  and  cooperation  between  students  encouraging  successive 
improvements  in  written  texts  as  a way  to  develop  writing  and  thinking  skills.  Curriculum  guidelines  also  advise  that 
teachers  must  find  learning  contexts  promoting  an  integrated  development  of  listen,  speaking,  reading  and  writing 
skills.  They  also  enhance  the  teacher  responsibility  in  creating  active  and  learner  centered  activities  where  processes, 
rather  products,  were  valued. 

These  general  guidelines  follow,  generally  speaking,  theoretical  assumptions  we  have  already  presented. 

Research  design 

This  study  was  designed  and  conducted  following  the  qualitative  paradigm,  which  fits  well  the  nature  of 
this  study.  Following  Strauss  and  Corbin  "qualitative  methods  can  be  used  to  uncover  and  understand  what  lies 
behind  any  phenomenon  about  which  little  is  yet  know  ...  and  can  give  the  intricate  details  of  phenomena  that  arc 
difficult  to  convey  with  quantitative  methods"  (p.  19). 

Wc  combined  several  approaches,  following  a trend  of  utilizing  mixed-methods  (Crcswell,  1994;  Grccnc, 
Caracelli,  & Graham,  1989;  Guba,  1992).  Although  wc  may  consider  our  research  as  ethnographic,  the  grounded 
theory  (Glaser  & Strauss,  1967;  Strauss,  1995;  Strauss  & Corbin,  1990)  was  the  main  approach  in  this  study. 

Intervention  Design 

We  have  selected  teachers  who  usually  create  learning  environment  that  follow  the  orientations  of 
theoretical  framework  of  this  study,  and  routinely  use  educational  technology  in  their  classrooms  for  several 
purposes. 

The  research  team  was  directly  involved  with  school  teachers.  Nevertheless,  wc  both  consider  that  teachers 
have  a central  role  and  the  main  responsibility  in  designing  learning  environments  where  educational  technologies 
are  integrated.  So,  our  role  as  researchers  is  not  to  orient  as  experts,  but  to  facilitate  and  support  teachers  work  in 
this  domain. 

The  study  was  conducted  during  two  school  years  in  four  elementary  (grades  one  and  two)  and  secondary 
schools,  involving  just  a classroom  and  a teacher  each  one.  It  included  mainly  writing  activities  with  the  word 
processor  but  also  videoconference  and  WWW  for  pre-writing  activities  and  other  purposes. 

Videoconference 

Five  videoconferences  were  carried  out:  two  for  research  team  and  school  teachers  to  plan  the  students' 
activities,  testing  the  connections  and  other  details;  three  integrated  in  pre-writing  students'  activities.  All  the 
videoconferences  were  implemented  between  University  of  Minho,  Portugal,  and  the  University  of  Exeter,  U.K. 
where  a colleague  of  us,  who  was  taking  his  PhD  there,  helped  to  set  up  the  sessions. 

Secondary  students  came  from  a professional  course  in  media  communication,  so  their  aim  was  to  know,  in 
a hands-on  way,  how  videoconference  system  works  and  its  role  in  actual  media  communication  field.  From  Exeter, 
our  colleague  demonstrated  the  functioning  of  videoconference  system  and  other  multimedia  tools  and  services. 
They  also  discussed  implications  of  videoconference  in  social,  personal  and  educative  domains. 

Videoconference  sessions  for  elementary  students  followed  a different  approach.  Students  developed 
project  work  in  class  comparing  some  aspects  of  society  and  culture  in  both  countries  to  exchange  ideas  about  that. 
As  their  colleagues  did,  they  had  previously  prepared  a set  of  topics,  questions  and  other  materials,  like  pictures, 
cards  and  written  slogans,  to  support  the  presentation  of  their  project  work.  Colleagues  who  did  not  participate  in 
videoconference  engaged  in  other  kind  of  activities  integrated  in  their  global  project  work. 
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As  in  videoconference,  the  intention  of  secondary  students  was  to  know  how  Web  works  rather  than  using 
it  as  a way  of  gathering  specific  information.  The  aim  of  preparatory  students  was  to  obtain  information  related  to 
their  specific  subject  of  project  work  they  usually  do.  The  class  was  split  in  groups.  Each  group  participated  in  three 
3 hour  sessions. 

In  the  first  session,  and  after  an  initial  period  of  free  web  browsing,  students  conducted  web  searches 
according  to  their  interests.  They  had  previously  chosen  a number  of  topics  and  key  words  in  English,  which  were 
used  to  search  for  information.  They  were  allowed  to  choose  navigation  itineraries.  In  order  to  prepare  the  second 
session,  a number  of  selected  web  sites  were  bookmarked.  The  information  retrieved  was  printed  out  and  studied 
during  the  week  as  homework  and  in  class.  With  teacher  support,  topics  for  further  exploration  in  the  second  session 
were  selected. 

In  the  second  session  the  collection  of  information  was  completed.  A vast  amount  of  topics  followed  by  an 
abstract,  in  English,  resulted  from  this  process.  Students  then  analyzed  this  information:  they  identified  the  issues 
emerging  and  selected  topics,  which  were  translate^  into  Portuguese.  After  that,  they  discussed  and  structured  their 
ideas  and  outlined  a plan  of  a text  to  be  written  later.  The  role  of  the  teacher  was  fundamental  in  this  phase,  in  order 
to  help  students  to  select,  organize  and  structure  the  available  information.  Finally,  students  wrote  texts  in  a 
collaborative  way  and  using  word  processor. 

The  third  session  occurred  the  next  school  year  and  was  similar  to  the  first  ones,  as  only  the  project  subjects 

changed. 

Word  processor 

In  students  writing  we  considered  pre-writing,  writing  and  rev  is  ion -correct  ion  phases.  Although 
videoconference  and  web  activities  had  other  aims,  they  were  also  considered  as  pre-writing  activities.  They 
intentionally  included  different  curricular  subjects  as  a way  to  promote  an  interdisciplinary  approach.  As  referred  by 
Kaufman  (1994)  it  is  absurd  that  teachers  promote  writing  tasks  only  as  a means  to  evaluate  students'  ability  to 
write.  Promoting  writing  activities  in  several  curricular  domains  was  a natural  way  to  practice  and  improve  writing 
skills. 

Written  activities  with  word  processor  were  carried  out  at  schools  in  various  contexts,  including  language 
classes.  Students  were  randomly  organized  in  groups.  They  wrote  as  they  usually  did.  No  suggestions  about  writing 
process  were  given  to  students.  Texts  produced  had  a real  audience  because  they  were  integrated  in  several 
communicative  circuits,  like  school  newspapers,  project  work  and  exhibitions. 

Data  collection 

Data  were  collected  by  different  methods  and  from  different  sources:  classroom  observation,  students' 
interviews  and  teacher  description,  evaluation  and  comments  to  the  activities  carried  out. 

Data  triangulation  increased  the  validity  of  the  study  (Cohen  & Manion,  1 1989;  Dcnzin,  1978;  Foster, 
1996;  Marshall  & Rossman,  1989;  Maxwell,  1996;  Scott,  1996)  and  help  to  grasp  the  complexity  of  the  research 
area. 

The  main  data  source  were  a series  of  videotapes  on  videoconference.  WWW  and  word  processor  activities 
above  presented.  There  was  a tot of  40  hours  of  observation.  All  videotapes  were  transcribed. 

Wc  interviewed  students  informally.  Students  had  to  respond  to  both  general  and  specific  questions  about 
activities  they  carried  out  with  technologies.  We  also  presented  some  open-ended  questions  designed  to  give 
students  the  opportunity  to  share  their  perceptions  and  approaches  and  to  explore  their  thinking  not  only  about  task 
earned  out  but  also  about  future  work.  All  interviews  were  videotaped  and  transcribed. 

Teachers  were  also  asked  to  give  their  opinions  about  work  carried  out  with  technologies  by  means  of 
informal  talks  at  the  end  of  each  session  and  by  written  commentaries. 

Data  analysis 

The  process  of  data  analysis  is  still  in  course  and  it  began  during  dala  collection  itself,  where  some  general 
patterns  began  to  emerge.  In  the  next  step,  all  observations  and  interviews  were  transcribed  as  completely  and 
literally  as  possible  with  some  essential  prosodic  features. 

These  transcripts  arc  being  analyzed  using  the  grounded  theory  methods  (Glaser  & Strauss,  1967;  Strauss, 
1995;  Strauss  & Corbin,  1990).  Theory  will  be  inductively  built  using  well-established  steps.  The  first  procedure  of 


open  coding  is  breaking  down  and  conceptualizing  data.  T patterns  that  emerged  were  then  categorized  and 
potential  labels  are  being  explored. 

The  next  analytic  procedure,  axial  coding , involves  linking  subcategories  to  a category  by  tracing  relations 
between  them  in  terms  of  causal  conditions , phenomena  studied , context , intervening  conditions , action/interaction 
strategies , and  consequences.  This  paradigm  model  shows  interdependency  of  categories 

Selective  coding  procedures  shall  lead  us  to  the  conceptualization  of  the  central  phenomenon,  the  core 
category , around  which  all  the  other  categories  arc  integrated.  This  is  the  core  message  of  the  research. 

These  themes  and  emerging  trends  are  presented  as  proposition  statements. 

As  a matter  of  fact,  many  authors  (Lee  & Ficldind,  referred  by  Prothero,  1996;  Miles  & Hubcrman,  1994;) 
suggest  that  qualitative  researchers  can  not  ignore  computer  software  for  qualitative  data  analysis.  According  to 
Richard  and  Richard  (cited  by  Prothero,  1996)  "the  computer  allows  researchers  to  explore  and  interrogate  emerging 
patterns,  to  keep  asking  questions  and  drawing  together  different  levels,  contexts,  relationships  to  test  their 
significance  and  to  chase  ideas  about  the  ways  they  pattern  data". 

Our  choice  for  qualitative  data  analysis  software  was  Nud*ist  (Non-numcrical  Unstructured  Data  Indexing, 
Searching  and  Theorizing).  This  is  a multi-functional  software  system  that  helps  researcher  to  handle  non-numcrical 
unstructured  data  by  indexing  searching  and  theorizing. 

Up  to  the  present  moment,  we  have  identified  and  broadly  described  the  key  features  of  some  emergent 
patterns.  In  this  first  step  of  data  analysis  we  seek  to  identify  emergent  "voices"  still  without  their  articulation  and 
integration.  According  to  the  title  of  this  paper,  wc  just  report  here  these  "voices  from  experience"  in  a preliminary 
framework  of  research  findings. 

Research  findings  and  discussion 

Findings,  although  provisory,  as  wc  have  clearly  stated,  allow  us  to  enhance  some  of  the  key  emergent 
voices  and  to  realize  that,  often,  they  tunc  with  voices  from  other  research  studies. 

1 . Videoconference  promotes  the  connection  between  people  at  various  levels  and  in  several  ways. 

The  cyberspace  "that  invisible  arena  where  humans  arc  connected  by  electronic  technology,  not  by 
geographic  proximity"  (Anderson-Inman,  1996,  p.  134)  allows  a climate  very  similar  to  face-to-facc  communication 
which  creates  a real  need  to  express  their  ideas  and  to  listen  and  understand  colleagues  in  a really  interactive  way. 

Students  from  different  places  and  with  different  backgrounds  and  capabilities  work  together  discussing 
personal  ideas  and  sharing  information  about  different  curricular  subjects  in  a vivacious  and  authentic  interpersonal 
communication.  This  collaboration  not  only  allows  different  perspectives  in  treating  subjects,  but  also  increases 
students’  oral  skills.  Students  must  think  and  react  rapidly  and  this  spontaneity  added  to  peer  support  stimulates 
some  interventions  of  less  skilled  students  that  usually  do  not  take  this  risk.  In  fact  wc  noticed  remarkable 
interventions  and  even  "Hash  of  eloquence"  of  special  needs  students  who  usually  were  silent  in  other  classroom 
activities.  It  seems  that  colleagues  support  and  communication  atmosphere  itself  increased  their  self-confidcnce 
leading  them  to  overcome  a bit  their  affective  and  linguistic  difficulties. 

In  such  an  authentic  and  dynamic  situation,  where  students  must  find  quickly  and  efficiently  what  they 
need,  it  was  easily  accepted  both  by  teachers  and  by  students  that  speech  was  not  always  correct.  However,  from 
time  to  time,  they  asked  for  help,  reformulated  speech  and  they  were  spontaneously  corrected  and  supported  by 
colleagues  which  reveals  their  interest  in  improving  their  language  skills.  Giving  immediate,  contextual  and  friendly 
feedback  seems  to  be  a good  way  to  articulate  theory  and  practice  in  language  learning. 

We  noticed  that,  in  spite  of  students'  enthusiasm  and  happiness  to  "meet  in  virtual  classrooms,  collaborating 
and  co-constructing  their  educational  experience"  (Anderson-Inman,  1996,  p.  137),  their  best  pleasure  will  be  to 
meet  personally  their  new  British  friends.  It  was  clear  that  videoconference  does  not  replace  or  diminish  face-to-facc 
relationships  that  arc  essential  in  education  as  in  life;  rather  it  creates  in  students  the  need  to  know  better  people  they 
interacted  with. 

One  of  the  major  challenges  in  foreign  language  learning  is  to  create  environments  where  students  arc  so 
immersed  in  target  language  contexts,  that  they  really  need  to  communicate  with  others.  Such  an  authentic 
environment  encourages  the  discovery  of  language  as  tool  for  real  communication  purposes.  Evidence  we  have 
collected  suggest  that  videoconference  offers  a great  opportunity  for  it. 

2.  The  way  the  information  is  provided  on  the  Web,  following  a hypermedia  structure  hut  with  a dominant 
amount  of  text  provides  opportunities  for  the  development  of  written  comprehension.  Wc  noticed  that  students  read 


attentively  in  order  to  make  further  choices.  This  hypcricxi/hypcrmedia  model  also  required  the  development  of  new 
reading  techniques:  due  to  the  amount  of  information  they  arc  faced  with,  students  quickly  discover  reading  must  be 
selective  and  oriented  to  their  interests.  So,  they  arc  forced  to  abandon  the  sequential  reading  way  they  arc  used  to 
and  adopt  pro-active,  flexible  and  focused  reading  and  selection  strategies. 

Web  based  activities  provide  an  opportunity  to  improve  linguistic  skills  both  in  Portuguese  and  in  English 
because  students  constantly  translate  between  these  two  languages  either  to  input  search  queries  or  to  understand 
search  results. 

Gathering  information  in  Web  also  promotes  the  development  of  the  thinking  skills  required  by  information 
search,  selection,  analysis,  evaluation  and  synthesis  - these  abilities  are  becoming  increasingly  important  nowadays. 
Students  outlined,  discussed  and  refined  search  strategies  seeking  for  more  precision.  Given  the  quantity  and  variety 
of  information  collected,  students  gathered  a wide  variety  of  perspectives  and  ideas  with  a range  of  levels  of 
complexity.  According  to  Ryder  & Graves  the  "Information  age  literacy  requires  learners  who  arc  proficient  in 
focusing  attention,  attending  to  an  accessible  and  dynamic  medium,  and  thinking  critically"  (1997,  p.  251).  Then,  in 
a synthesis  process,  students  compared,  contrasted  and  structured  the  information  under  a coherent  framework: 
"Information  literacy  requires  learners  to  obtain  and  derive  meaning  from  diverse  and  extensive  sources  of 
information  and  to  regulate  their  own  thinking  in  pursuit  of  a level  of  understanding  that  goes  well  beyond 
memorizing  the  details  of  a classroom  textbook"  (Ryder  & Graves,  1997,  p 252).  Carrying  out  these  tasks  requires 
skills  classified  by  Bloom  at  the  top  of  the  hierarchy  of  cognitive  objectives. 

Evaluating  validity  of  the  information  available  in  web,  on  one  hand,  and  selecting  the  most  important 
topics,  on  the  other  hand,  are  probably  the  biggest  and  more  complex  challenge  both  for  teachers  and  for  students.  In 
fact,  as  stressed  by  Roberts  "students  need  to  evaluate  the  content  of  what  they  find  on-line,  just  as  they  do  with 
traditional  sources.  Evaluating  sources  and  comparing  facts  and  opinions  arc  essential  activities  when  exploring  on- 
line databases".  But,  the  same  authors  remark,  "How  can  children  learn  to  evaluate  the  information  they  obtain?" 
(1990,  p.79). 

Since  students  worked  in  groups,  they  had  the  opportunity  to  share  ideas,  achievements  and  difficulties, 
both  in  content  domains,  searching  features  and  linguistic  area.  This  kind  of  task  in  such  a context  provides  support, 
encouragement,  and,  as  asserted  by  Oliver  (1997),  the  development  of  students’  social  and  thinking  skills. 

According  to  Roberts  "just  as  the  world  is  exploding  with  new  information,  it  is  also  becoming  increasingly 
complex.  Today,  answering  questions,  solving  problems,  and  exploring  new  ideas  requires  that  people  work 
together....  This  collaboration  requires  communication,  communication  with  people  in  next  office,  in  another  city, 
or  around  the  world"  (1990,  p.3).  Once  again,  this  emphasis  in  collaboration  and  communication  demonstrates  how 
useful  are  Web  based  activities  for  (he  present  and  future  life  of  students. 

With  Ryder  and  Graves  we  consider  that  "[tjhcre  is  no  doubt  that  if  students  arc  to  be  effective  in  their 
ability  to  engage  in  the  higher  level  thinking  tasks  associated  with  information  literacy,  they  will  need  to  regularly 
engage  in  these  tasks  and  acquire  strategies  tailored  to  the  diverse  a;  \ ever-changing  resources  on  the  Internet" 
(1997,  p .253).  Web  activities  in  cooperative  learning  environments  arc,  w c believe,  a good  step  in  this  direction. 

3.  Writing  activities  in  several  curricular  domains  are  a good  way  to  improve  writing  skills,  and  provide 
students  with  an  opportunity  to  approach  language  not  only  as  an  object  of  study  in  itself  but  as  a learning  tool  in 
other  curricular  domains.  In  addition,  the  fact  that  texts  were  written  for  a real  purpose  and  had  a real  audience  also 
increases  students  interest  in  the  task. 

Videoconference  and  Web  based  activities  are  good  pre-writing  strategics  not  only  because  of  students' 
gains  in  terms  of  language  and  curricular  content  we  referred  above,  but  also  as  a way  to  enhance  the  need  for 
selection  and  organization  of  ideas,  before  writing.  In  fact,  information  gathered  was  usually  so  copious,  rich  and 
disperse,  that  they  really  fell  the  need  to  organize  it. 

In  such  a context,  where  content  features  were  previously  prepared,  and  peer  interaction  encouraged,  word 
processor  increases  students  interest  in  the  writing  task  and  stimulates  constantly  improvement  of  their  text. 

Experience  clearly  shows  and  research  evidence  suggests  (Underwood,  1991)  that  the  use  of  traditional 
writing  toal.s  ir.ake^  it  difficult  to  improve  written1  texts,  since  it  is  necessary  to  repeat  the  writing  process  from  start. 

It  is  iiriportanl  that  students  do  not  force  themselves  to  be  correct  at  their  first  attempt.  The  process  of 
transforming  ideas  in  writing  is  hard  owing  to  the  various  demands  that  operate  simultaneously:  the  sequence  of 
ideas,  the  vocabulary,  the  orthography,  the  syntax,  the  logical  sequence  of  the  text.  We  can  think  of  few  things  at  a 
time,  so  attention  to  all  these  aspects  overloads  short  term  memory  and  they  tend  to  give  priority  to  formal  aspects 
rather  than  to  more  cognitive  challenging  requirements.  Since  word  processor  allows  easy  modifications  in  a drafted 


text,  students  can  pay  attention  to  each  aspect  at  a time:  after  expressing  ideas,  they  debate  until  they  arrive  to  the 
best  form. 

It  seems  that  because  it  is  so  easy  to  correct,  students  are  encouraged  to  make  constant  improvements  in  a 
first  and  free  draft:  they  write,  they  modify  as  much  as  they  need,  often  with  colleagues’  support,  and  the  text 
remains  clean. 

It  was  observed  that  corrections  students  did  within  or  after  text  production  were  dominated  by  spelling, 
punctuation,  accentuation  and  vocabulary.  It  goes  without  saying  that  we  are  not  surprised  with  their  intervention  at 
such  a low  level:  it  is  the  style  of  correction  teachers  usually  uses!  Obviously,  we  cannot  expect  students, 
spontaneously,  do  more.  Technology  by  itself  has  had  little  impact  on  student  achievement  (Dufficld,  1997). 

But,  if  the  teacher  follows  the  writing  process,  going  near  the  students  that  need  assistance,  hc/she  can 
understand  the  source  of  students'  doubts  and  mistakes  and,  in  consequence,  he  can  support  them  in  their  attempts  of 
correction.  Focusing  these  real,  specific  and  in  context  problems  is  a great  way  to  practice  grammar  not  as  an  end  in 
itself  but  as  a means  of  improving  written  communication.  By  doing  so,  and  according  to  MacEncry,  " [g]rammar 
learning  means  not  studying  and  memorizing  prescribed  rules  about  grammar,  but  acquiring  skills  and  insight  into 
the  ways  in  which  grammar,  that  is  to  say  the  forms  of  words  and  the  sequences  in  which  they  appear  in  sentences, 
paragraphs  and  texts,  communicates  meaning"  (1997,  p.9) 

In  addition  to  word  processor  performances,  it  seems  that  the  motivation  to  discuss  and  make  suggestions 
to  improve  early  versions  of  the  text  was  also  enhanced  by  the  cooperative  nature  of  the  task  as  well  as  the  common 
use  of  the  computer  screen,  the  "virtual  paper"  (SablE  and  Bouyssou,  1995).  As  a consequence  of  group  work  iri 
students  tend  to  think  aloud,  expressing  their  own  ideas  and  doubts,  asking  colleagues  for  what  they  do  not 
understand.  Each  one  made  his  contribution  according  to  his  own  capacities,  so  text  results  from  the  combined  effort 
of  them  all. 

According  to  Dale  (referred  by  Sanacore,  1997)  "the  real  value  of  co-authoring  groups  is  not  the 
collaborative  product  but  rathtji  the  peer  interaction.  As  the  students  talk  about  their  writings,  they  may  discover 
novel  ways  of  thinking  about  writing;  however,  this  valuable  growth  may  or  may  not  be  revealed  in  the  written 
product".  In  fact,  we  are  not  sure  that  texts  produced  in  word  processor  were  much  better  than  previous  ones,  but 
data  collected  demonstrated  that,  together,  word  processor  in  cooperative  learning  activities  and  teacher  support,  can 
really  contribute  to  improve  written  skills.  In  addition,  such  an  environment  of  speaking-rcading-writing  promotes  a 
comprehensive  development  of  all  these  skills. 

4.  Cooperative  learning  environments  not  only  reinforce  students'  spontaneous  disposition  to  share  and 
discuss  ideas  but  also  encourage  the  less  skilled  ones  to  participate  in  task,  in  any  way,  and  support  i heir  attempts. 
Naturally,  there  also  occur  some  disagreements  between  students  and  this  promotes  rich  and  usually  useful 
discussion.  Through  this  dialog,  they  learn  to  explain  and  to  specify  their  ideas  and  also  how  to  understand 
colleagues1  ideas  and  how  to  ask  for  more  explanations.  According  to  Bearison  (referred  by  King)  "resolving  such 
cognitive  conflicts  should  create  some  change  in  the  learner's  cognitive  structures  or  processes.  The  aggregate  of 
these  changes  in  group  member's  cognition  may  promote  the  group’s  success  in  solving  the  problem"  (1989,  p.l  1 ). 

Learning  from  one  another,  as  it  occurs  in  cooperative  work,  stimulates  students  engagement  in  their 
learning  process  thus  increasing,  as  is  pointed  out  by  current  constructivist  learning  theories,  their  responsibility  in 
constructing  their  own  knowledge.  This  is  an  important  step  in  preparing  students  for  the  challenges  of  their 
tomorrow,  since,  more  and  more,  one  constantly  needs  to  update  knowledge  and  abilities  in  several  domains  (Leu, 
1996). 

Constructivist  environments  do  not  reduce  teachers  responsibilities  in  learning  process,  rather  they  require 
"a  fundamental  change  in  the  way  teachers  approach  instruction  . . . (T)hc  highest  compliment  a teacher  can  receive 
from  a learner  is  you  made  this  so  easy  that  / was  able  to  learn  it  by  myself  (Mehlingcr,  1995,  pp.  76-77). 

5.  There  arc  no  easy  neither  definitive  answers  to  integrating  technology  into  educational  learning  contexts. 
Yet,  as  Brownsuggcsls,  "it  takes  about  30  years  to  blend  a new  idea  into  daily  practice.  In  teaching  new  ideas  and 
teaching  methods  cycle  in  and  out  favor  more  frequently.  Integrating  them  solidly  into  practice,  however,  may  still 
take  several  years"  (1997.  p.  17).  Because  we  believe  that  for  teachers  good  experiences  are  more  motivating  than 
good  theories  we  consider  that  research  must  be  rooted  inconcrete  classroom  activities. 

The  purpose  of  this  study  was  to  explore  the  contribution  of  educational  technologies  and  cooperative  work 
to  develop  linguistic  and  thinking  skills.  As  the  major  authors  of  grounded  theory  suggest,  research  should  be 
designed  as  a continually  evolutionary  process.  We  took  a first  step.  Further  data  analysis  will  give  a deeper 


understanding  of  the  problem  bearing  in  mind  that  "in  postmodern  realm  we  cannot  claim  lo  know  anything 
completely"  (Addison,  1997,  p.  107). 
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Introduction 

While  there  arc  numerous  books,  journal  articles,  and  models  of  instructional  design,  few  of  these  resources 
discuss  the  impact  that  environmental  forces  have  upon  decisions  relating  to  instruction  and  training.  There  are 
forces  wiihin  and  outside  of  educational  institutions,  businesses,  and  agencies  that  play  a significant  role  in  decisions 
of  whether  or  not  to  use  instruction  or  training.  Issues  relating  to  government  legislation,  insufficient  resources, 
incompatibility  with  societal  goals,  and  lack  of  strong  support  can  cause  instruction  or  training  to  be  ineffective. 
The  importance  and  high  cost  of  training  and  instruction  point  to  a need  for  instructional  designers  to  make  certain 
that  the  end  result  is  effective. 

In  this  paper  we  offer  a definition  for  the  word  effective  and  then  discuss  implications  of  research  on  the 
environment  in  relation  to  designing  effective  instruction.  Similarities  and  differences  involved  in  designing 
effective  instruction  are  presented  across  four  different  contexts:  K-12.  Higher  Education,  Human  Services,  and 
Corporate.  We  posit  that  the  primary  reason  instruction  and  training  fail  to  achieve  intended  goal(s)  is  due  to 
instruc'ional  designers’  tendency  to  overlook  or  underestimate  environmental  issues.  With  this  in  mind,  we  suggest 
that  environmental  issues  be  thoroughly  addressed  before  instructional  designers  proceed  with  the  design  process. 
Our  views  and  model  for  designing  effective  instruction  have  been  formed  as  a result  of  our  experiences  and  from  an 
examination  of  research  and  theory  from  a variety  of  fields. 

Definition  for  Effective 

The  word  effective  has  many  meanings.  Capable  of  producing  a desired  result  or  effect  is  one  of  its 
meanings.  Similar  definitions  include  “ready  for  service”  and  “prepared  for  use  or  action".  Wc  all  know  of  people, 
ideas,  and  technological  innovations  that  have  the  potential  to  accomplish  a desired  effect  or  result,  but  never 
advance  past  a state  of  readiness.  An  essential  part  of  being  effective  is  having  the  potential  or  the  ability  to  achieve. 
However,  this  docs  not  encompass  the  entire  sense  of  the  word. 

Another  meaning  associated  with  the  word  effective  is  the  actual  production  of  an  intended  or  desired 
outcome  or  result.  Words  related  to  this  meaning  include  operative,  working,  and  active.  Emphasis  in  this  meaning 
is  on  actual  use  for  achieving  results  whereas  emphasis  in  the  previous  meaning  is  on  potential  ability  of  someone  or 
something  to  produce  intended  outcomes. 

Wc  believe  that  the  definition  for  effective  should  include  both  potential  and  production.  The  definition, 
“producing  or  capable  of  producing  a desired  effect,”  found  in  both  Webster’s  College  Dictionary  (1990)  and  The 
American  Heritage  Dictionary  (1983)  implies  that  a separation  of  two  meanings  is  acceptable.  Ability  to  perform 
without  achieving  the  expected  results  is  an  ineffective  use  of  time,  talent,  and  other  resources.  Truly  effective 
individuals,  ideas,  and  innovations  are  not  only  capable  of,  but  actually  produce  an  intended  or  desired  effect. 

Application  of  Effectiveness  to  Instructional  Design 

When  a request  for  training  or  instruction  is  received,  most  instructional  designers  conduct  a needs 
assessment  to  examine  gaps  between  current  and  desired  human  performance  and  gather  information  that  assists 
with  problem  solution.  If  human  performance  deficiencies  are  identified,  designers  typically  begin  work  to 
determine  tasks  and  corresponding  knowledge,  skills,  and  attitudes  to  be  included  in  the  instruction  or  training.  In 
essence,  they  jump  right  into  planning  a solution  before  considering  many  of  the  essential  elements  required  to 
develop  instruction  that  is  effective. 

Instructional  design  models,  textbooks,  and  courses  focus  on  the  procedures  and  techniques  for  developing 
quality  instruction,  curriculum,  and  training.  Unfortunately,  this  becomes  the  sole  goal  for  most  novice  and  even 
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some  experienced  instructional  designers.  No  matter  how  well  <'  signed  an  instructional  product  might  be,  il  is 
ineffective  if  it  is  never  used. 

Effective  instruction  and  training  should  be  the  goal  for  all  instructional  designers.  If  our  argument  for  an 
all-encompassing  definition  for  effective  has  been  convincing,  then  it  should  be  clear  that  a well  designed 
instructional  product  alone  docs  not  guarantee  effectiveness.  In  addition  to  being  well  designed,  effective  training  or 
instruction  must  be  “operative”,  “active”,  or  “working”  to  eliminate  or  at  least  reduce  human  deficiency  problems. 
Based  on  our  knowledge  and  experience,  instructional  designers  arc  not  adequately  prepared  with  strategics  to  help 
ensure  that  the  latter  occurs.  Educational  institutions,  human  services  organizations,  and  businesses  arc  more  likely 
to  use  instruction  or  training  if  certain  issues  discussed  later  in  this  paper  are  addressed. 

Role  of  the  Stakeholders  and  Setting  in  Instructional  Design 

Creating  effective  instruction  or  training  requires  instructional  designers  to  perform  tasks  beyond  the  design 
of  quality  products.  Almost  all  instructional  and  training  products,  from  the  most  simple  to  the  most  complex,  are 
designed  to  be  used  in  a particular  setting  or  environment.  Therefore,  instructional  products  must  be  appiopriatc  lor 
the  setting  in  which  they  will  be  used.  In  addition,  the  implementation  of  instructional  and  training  products  is 
impacted  by  the  stakeholders,  people  who  can  affect  or  are  affected  by  its  implementation.  Many  instructional 
designers  underestimate  the  influence  of  stakeholders  and  the  setting. 

Examination  of  the  role  that  stakeholders  and  the  setting  have  in  the  instructional  design  process  commonly 
begins  and  ends  during  the  needs  assessment.  The  possible  causes  of  deficient  performance  can  be  attributed  to  lack 
of  knowledgc/skills,  lack  of  motivation/incentives  or  environmental  barriers  (Harless,  1978;  Rossett,  1991).  To 
determine  possible  causes  or  contributing  factors,  the  setting  is  examined  for  insufficient  resources,  deficient 
equipment,  deficient  process,  conflicting  demands/standards,  and  other  Haws.  Also  stakeholders  arc  consulted  and 
observed  to  gather  information  concerning  performance  problems.  If  the  setting  is  in  order,  instructional 
technologists  rule  it  out  as  a possible  cause  of  deficient  performance.  If  the  setting  is  flawed  in  some  way,  they 
make  recommendations  to  fix  it.  In  cither  case,  most  instructional  technologists  do  not  think  further  about  the  role 
that  the  stakeholders  or  the  setting  play  in  decisions  whether  or  not  to  use  instruction  or  training. 

Organizational  change  theory  has  long  noted  the  impact  that  events,  resources,  various  people,  and  their 
power  bases  can  have  on  whether  an  idea,  product,  or  technological  innovation  is  used  in  a setting.  Many  journal 
articles  and  books  providing  practical  suggestions  to  help  bring  about  change  in  elementary  and  secondary  schools, 
colleges  and  universities  or  businesses  can  be  found  in  the  literature  (Berman  & McLaughlin,  1978;  Fullan.  1982; 
Havelock,  1973;  Havelock  & Hubcrman,  1978;  Mayhew,  1976;  Rogers,  1995;  Zaltman,  Duncan,  & Holbcek,  1973; 
■Zaltman,  Florio,  & Sikorski,  1977).  These  publications  offer  good  suggestions  usually  in  Lhe  form  of 
recommendations  and  guidelines,  but  the  advice  is  nonspecific  to  instructional  design. 

Drawing  on  organizational  change  theory  Diamond  (1989)  emphasized  the  importance  of  considering 
institutional  priorities,  faculty  support,  and  other  environmental  issues  in  his  development  model.  When  designing 
or  redesigning  a course  or  curriculum  Diamond  (1989)  suggests  that  instructional  designers  do  the  following: 

Before  a project  is  begun,  a number  of  factors  must  be  consider:  (1)  how  important  the  project  is  to  the 
academic  department,  school,  or  college,  and  when  appropriate,  even  to  the  institution,  (2)  how  the  project 
will  be  received  by  others,  (3)  whether  the  necessary  support  is  available,  and  (4)  if  the  required  faculty 
commitment  will  be  made  (p.  21). 

Although  the  Diamond  model  was  developed  for  higher  education,  similar  environmental  impacts  affect 
instruction  in  other  settings.  Instructional  designers,  regardless  of  the  setting,  should  understand  what  environmental 
elements  are  likely  to  impact  training  or  instruction  and  how  they  will  affect  it.  This  requirement  for  an 
understanding  of  environmental  issues  goes  beyond  the  concepts  and  practices  that  arc  traditionally  taught  in  the 
instructional  design  field. 

Model  for  Creating  Effective  Instruction 

Our  experiences  indicate  that  while  well  designed  instructional  products  arc  an  essential  factor,  usage  is 
largely  dependent  upon  the  appropriateness  of  the  instruction  for  the  respective  stakeholders  and  setting.  Therefore, 
instructional  designers,  regardless  of  the  setting,  should  understand  and  address  elements  that  arc  likely  to  impact 
training  or  instruction.  When  issues  relating  to  the  setting  and  stakeholders  arc  left  unaddressed,  the  result  may  be 
losses  of  time,  personnel,  money,  and  other  resources.  Neither  schools,  universities,  businesses,  nor  human  service 
agencies  can  afford  such  losses. 


A model  of  E =PxSxS  will  be  used  to  frame  the  discussion  about  developing  effective  instruction  or 
training  (see  Figure  1).  Respectively,  the  letters  stand  for  effective,  product,  stakeholders,  and  setting.  The 
multiplication  symbol  is  used  between  letters  rather  than  an  addition  symbol  because  if  any  factor  is  missing  the  end 
product  will  be  zero  or  ineffective.  This  model  is  generic  in  the  sense  that  instructional  technologists  in  any  setting 
can  use  it.  In  addition,  novice  instructional  technologists  will  find  the  model  easy  to  comprehend,  while  most 
experienced  designers  will  appreciate  having  guidelines  and  suggestions  for  addressing  stakeholder  and  setting 
issues. 


Eteurc  I . E=PxSxS:  A Model.  for  Designing  Effective  Instryclion  or  Training 


Product 

With  the  rise  of  “quality*’  management  programs,  many  organizations,  including  educational  ones,  have 
become  more  concerned  about  improving  the  quality  of  products  at  all  levels  (Rossett  and  Krumdieck,  1992). 
“Quality  Products”  in  this  sense  means  “fitness  for  use”  or  “conforming  to  specifications".  Designers  satisfy  this 
standard  by  demonstrating  credibility,  striving  for  the  "ideal"  instruction,  producing  the  promised  deliverables, 
delivering  instruction  on  time  and  within  budget,  and  conducting  formative  and  summative  evaluations.  The  design 
and  development  of  quality  instructional  products  is  the  primary  goal  of  many  designers.  Therefore,  designers  are 
well  aware  of  the  standards  for  achieving  quality  products  illustrated  in  design  models  and  discussed  in  texts  and 
articles. 


Suggested  Questions  Pertaining  to  Product 

♦ Are  the  deliverables  (products  and/or  services)  appropriate  and  sufficient  for  addressing  the  deficiency 
problem? 

♦ Do  you  or  other  team  members  have  the  necessary  skills  to  design  and  develop  the  instruction  or 
training? 

4 Has  the  developed  instruction  or  training  been  field  tested  to  see  if  it  actually  eliminates  human 
performance  deficiencies? 

Stakeholders 

Achieving  commitment  and  support  from  stakeholders  gets  less  consideration  in  the  realm  of  instructional 
Jcsign  than  docs  the  design  and  development  of  quality  products.  Typically,  the  designer  gets  a contract  and  jumps 
into  designing  instruction.  Time  spent  on  gaining  the  support  of  others  within  the  institution  or  business  is  often 
forgotten  until  it  becomes  a problem.  If  commitment  and  support  issues  are  dealt  with  early  on  it  is  believed  that 
success  and  effectiveness  will  increase.  Most  instructional  designers  can  relay  horror  stories  of  working  hard  on  a 
project  requested  by  an  organization  leader  or  decision  maker  only  to  find  that  half  way  through  the  project  a 
powerful  mid-level  opinion  leader  or  group  of  opinion  leaders  arc  not  in  support  or  agreement  on  the  project.  Part  of 
addressing  acceptance  issues  i paying  attention  to  the  top  decision-makers  and  opinion  leaders  within  the 
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institution.  Winning  the  support  of  these  people  can  mean  the  difference  between  a product  that  sits  in  a binder  on  a 
shelf  and  one  that  is  tom  and  tattered  from  use.  The  designer  should  also  collaborate  with  others,  especially  SMH's 
(subject  matter  experts)  and  the  intended  recipients  of  the  training.  The  input  that  SME's  and  recipients  can  provide 
will  smooth  the  road  toward  implementation. 

Suggested  Questions  Pertaining  to  Stakeholders 

♦ Has  the  need  and  logic  of  the  training  or  instruction  been  communicated  to  the  stakeholders  (people 
who  can  affect  or  will  be  affected  by  its  implementation?) 

♦ Is  stakeholder  participation  sought,  welcomed,  and  their  advice  used? 

♦ Do  key  decision-makers  support  the  instruction  or  training? 

♦ Do  the  institution's  opinion  leaders  support  the  instruction  or  training? 

♦ Is  union  support  necessary  for  the  training  or  instruction  to  be  conducted,  and  if  so  has  it  been 
received? 

Setting 

Analysis  of  the  setting  for  which  the  instruction  or  training  is  being  designed  is  often  under  emphasized. 
Setting  refers  to  the  physical  environment,  the  culture,  and  the  resources  of  the  institution  or  organization.  Setting 
can  also  involve  such  things  as  the  technological,  social,  political,  and  economic  conditions  or  trends. 

Suggested  Questions  Pertaining  to  the  Setting 

♦ Do  you  have  an  understanding  of  the  culture  that  guides  the  school,  university,  business,  or  other 
institution  in  which  the  instruction  or  training  is  intended? 

♦ Can  resources  required  for  the  training  or  instruction  be  acquired  and/or  developed? 

♦ Are  facilities,  equipment,  and  other  items  necessary  to  conduct  the  training  or  instruction  available 
when  needed? 

♦ Is  the  instruction  or  training  aligned  with  technological,  organizational,  and  societal  trends? 

The  suggested  questions  are  fairly  comprehensive  but  may  need  to  be  modified  to  fit  your  particular 
situation.  Based  on  our  own  experiences  and  interviews  with  colleagues  we  propose  the  following  generalizations 
with'  regards  to  the  development  of  quality  products,  identification  of  stakeholders,  and  issues  involving  the  setting 
for  the  contexts  of  K-12,  Higher  Education,  Corporate,  and  Human  Services.  These  suggestions  are  based  on 
preliminary  discussions  and  will  be  backed  up  by  more  intensive  research  in  the  near  future. 

Our  experience  and  investigation  has  led  us  to  propose  that  in  K-12  contexts  the  perception  of  quality 
concerning  products  hinges  on  relevance  and  applicability  of  the  instruction  or  training  in  the  classroom.  When 
working  in  the  K-12  context  it  is  important  to  involve  teachers,  students,  principals,  and  superintendents.  Typically, 
K- 12  teachers  have  to  feel  a sense  of  ownership  before  they  will  implement  new  lessons  or  other  types  of  instruction 
in  the  classroom.  To  achieve  this,  their  involvement  is  necessary  at  all  levels  of  the  design  and  development 
process.  Finally,  the  K-12  setting  poses  some  significant  time  constraints  as  well  as  possible  physical  environment 
issues,  especially  in  elementary  schools.  If  instructional  designers  are  not  sensitive  to  limited  budgets  and  teachers’ 
scheduling  issues,  their  instruction  will  be  ineffective  because  it  will  not  be  used.  In  Higher  Education  contexts  we 
found  that  the  perception  of  a quality  product  is  often  tied  to  the  credentials  of  the  instructional  technology 
professionals  and  theoretical  grounding.  Some  of  the  stakeholders  who  are  important  to  involve  include  deans, 
department  chairs,  professors  and  students.  It  is  also  important  to  understand  the  culture  of  the  environment,  how 
people  work  together,  and  students'  learning  styles  and  preferences.  Human  Services  agencies  tend  to  equate  the 
quality  of  a product  with  the  emotional  reaction  of  those  involved  as  well  as  the  relevance  of  the  product  to  the 
needs  of  the  agency.  Diverse  perspectives  must  be  sought  from  staff  and  clients  in  order  for  instruction  or  training 
to  be  effective  in  this  setting.  Finally,  the  economic  constraints  and  culture  of  the  setting  are  very  important  to  the 
success  of  training  or  instruction  in  Human  Services  institutions.  The  last  context  discussed  in  this  concept  paper  is 
Corporate.  Corporate  clients  usually  need  to  see  a change  in  attitudes,  skills,  or  behavior  to  consider  training  or 
instruction  effective.  Many  corporate  clients  take  quality  a step  further  and  demand  measurable  results  tied  to  the 
bottom  line.  Top  management,  unions  representatives  (if  applicable),  and  the  individuals  affected  by  the  training  or 
instruction  must  be  involved  in  the  design  and  development  processes.  Finally,  the  culture  of  the  corporate  setting  is 
extremely  important.  Instructional  technology  professionals  must  investigate  the  culture  and  economics  of  the 
setting  if  they  arc  going  to  be  successful. 
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Conclusion 

Most  instructional  designers  faced  issues  of  product  quality,  stakeholder  acceptance,  and  appropriateness 
for  the  setting.  All  three  must  be  considered  in  appropriate  detail  if  designers  expect  to  be  effective.  The  first  step 
in  producing  effectively  designed  instruction  is  to  follow  a systematic  design  process  such  as  those  suggested  by 
Gagne,  Briggs,  and  Wager  (1992)  or  Dick  and  Carey  (1990).  In  addition  to  these  systematic  processes,  designers 
should  add  to  their  repertoire  of  activities  a more  intense  consideration  of  the  stakeholders  and  setting  for  which  the 
instructional  is  being  designed.  As  educational  institutions,  businesses,  and  other  organizations  face  increasing 
pressures  to  be  more  accountable  for  costs,  instruction  or  training  that  fails  to  achieve  desired  results  is  looked  upon 
with  greater  disfavor.  A focus  on  quality  along  with  stakeholders  and  setting  issues  will  improve  the  probability  of 
instructional  designers’  success  with  creating  effective  training  or  instruction. 
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Introduction 

The  need  for  reform  in  mathematics  education  is  well  documented  (International  Association  for  the 
Evaluation  of  Educational  Achievement,  1996;  National  Commission  on  Excellence  in  Education,  1983;  National 
Council  of  Teachers  of  Mathematics,  1989,  1991,  1995).  To  address  this  need,  the  National  Council  of  Teachers  of 
Mathematics  (NCTM),  in  concert  with  other  professional  organizations,  developed  the  NCTM  Standards  (National 
Council  of  Teachers  of  Mathematics,  1989,  1991,1995).  These  Standards  redefined  the  parameters  of  mathematical 
literacy,  the  scope  of  mathematics  instruction,  and  the  areas  of  content  emphasis  in  the  K-12  mathematics 
curriculum.  The  implications  of  these  changes  for  the  classroom  teacher  arc  considerable;  compliance  with  the 
Standards  may  require  the  classroom  teacher  to  reevaluate  his  or  her  role  and  the  roles  of  his  or  her  students  in  the 
tcaching/learning  process.  Modifying  classroom  roles  and  monitoring  the  resulting  changes  may  prove  to  be  a 
formidable  undertaking. 

The  Professional  Standards  for  Teaching  Mathematics  (National  Council  of  Teachers  of  Mathematics, 
1991)  identified  five  major  pedagogical  shifts  that  are  necessary  to  achieve  the  NCTM  goals  in  today's  mathematics 
classroom.  These  shifts  will  directly  affect  not  only  the  classroom  environment,  but  also  the  roles  of  students  and 
teachers  in  the  learning  process.  The  1991  Standards  proposed: 

• a shift  toward  classrooms  as  mathematical  communities  - away  from  classrooms  as  simply  a collection 
of  individuals; 

• a shift  toward  logic  and  mathematical  evidence  as  verification  - away  from  the  teacher  as  the  sole 
authority  for  right  answers; 

• a shift  toward  mathematical  reasoning  - away  from  merely  memorizing  procedures; 

• a shift  toward  conjecturing  and  problem  solving  - away  from  an  emphasis  on  mechanistic  answer- 
finding; 

• a shift  toward  connecting  mathematics,  its  ideas,  and  its  applications  - away  from  treating  mathematics 
as  a body  of  isolated  concepts  and  procedures,  (p.3) 

The  Curriculum  and  Evaluation  Standards  for  School  Mathematics  (National  Council  of  Teachers  of 
Mathematics,  1989)  pointed  out  that  advances  in  technology  have  extended  the  boundaries  of  mathematics  and 
underscored  the  importance  of  the  integration  of  technology  in  the  mathematics  curriculum.  In  a summary  of  the 
Thir*  Tntcrnational  Mathematics  and  Science  Survey,  Beaton,  Mullis,  Martin,  Gonzalez,  Kelly,  and  Smith  (1996) 
assc...J  that  technology  is  a key  factor  in  determining  the  economic  health  of  a country  They  also  pointed  out  that 
the  overriding  presence  of  technology  makes  new  demands  on  educational  communities  to  prepare  students  with 
high  levels  of  technical  competence  and  flexible  thinking.  For  these  reasons  the  International  Association  for 
Evaluation  of  Educational  Achievement  (IEA)  (1996)  recommended  appropriate  integration  of  technology  in  the 
mathematics  curriculum.  The  Board  of  Governors  of  the  Mathematical  Association  of  America  (MAA)  maintained 
that  computing  has  a profound  and  permanent  impact  on  the  practice  of  mathematics  at  all  levels  (Steen,  1992).  The 
MAA  recommended  that  technology  be  incorporated  into  the  mathematics  curriculum  in  a way  that  permits  students 
to  connect  mathematics  witv  diverse  fields  and  applications,  and  to  put  mathematical  ideas  into  action. 

The  NCTM  Standards  do  not  imply  that  the  use  of  technology  guarantees  mathematical  literacy,  or  that  the 
use  of  calculators  eliminates  the  need  for  students  to  learn  algorithms.  Rather,  the  standards  indicate  how 
technology  can  be  used  to  facilitate  the  learning  process  and  to  emphasize  the  fundamental  mathematics  students 
will  need. 

The  NCTM  Curriculum  and  Evaluation  Standards  staled  three  overreaching  goals:  (a)  students  should 
become  mathematical  problem  solvers;  (b)  students  should  learn  to  communicate  mathematically;  and  (c)  students 


should  learn  to  reason  mathematically  (pp.  5-6).  These  goals  assume  active  student  learning,  and  emphasize  the 
necessity  for  students  to  develop  an  understanding  of  mathematical  models,  structures,  and  simulations  so  that  they 
may  apply  their  mathematical  knowledge  flexibly  across  situations  and  disciplines.  In  this  context,  mathematics 
instruction  becomes  process-driven  rather  than  product-driven,  and  in  many  respects  becomes  a collaborative 
endeavor. 

A report  by  the  Council  of  Chief  State  School  Officers  (CCSSO)  on  state-by-state  trends  in  teaching 
practices  and  content  emphases  in  instruction  (Blank,  & Grucbel,  1995)  presented  data  representing  the  extent  to 
which  mathematics  classes  in  grades  '4  and  8 included  weekly  instructional  practices  that  are  consistent  with  the 
NCTM  standards.  The  four  types  of  instructional  practices  were:  (a)  work  in  small  groups,  (b)  write  reports  or  do 
mathematics  projects,  (c)  use  manipulative  objects,  and  (d)  use  calculators  in  class.  The  CCSSO  survey  found  that 
on  a weekly  basis  65%  of  the  mathematics  classes  nationally  have  students  work  in  small  groups,  1%  of  the  classes 
include  math  projects  and/or  reports,  46%  employ  math  manipulatives  with  instruction,  and  18%  use  calculators. 
These  findings  suggest  that  change  is  still  needed  in  many  United  States  classrooms  in  order  for  mathematics 
instruction  to  comply  with  the  NCTM  standards. 

The  Professional  Standards  for  Teaching  Mathematics  and  the  Assessment  Standards  for  School 
Mathematics  (National  Council  of  Teachers  of  Mathematics,  1995)  recognized  that  teachers  are  primary  figures  in 
changing  the  ways  in  which  mathematics  is  taught  and  assessed  in  schools,  and  that  technology  is  an  important 
component  of  that  change.  It  becomes  apparent  that  teachers  and  other  educators  must  bear  the  major  responsibility 
for  realizing  the  NCTM  Standards  in  practice. 

Strategies  and  programs  for  improving  mathematics  instruction  should  be  derived  from  sound  educational 
theory.  The  sociocultural  learning  theories  of  L.  S.  Vygotsky  (1986)  may  offer  guidance  in  developing  technology- 
based  mathematics  curriculum  materials  consonant  with  the  NCTM  goals  and  objectives.  Vygotsky's  theories, 
especially  when  they  are  applied  to  the  use  of  current  and  emerging  technologies,  have  the  potential  to  help 
classroom  teachers  narrow  the  gap  between  current  practices  and  the  vision  of  mathematics  education  represented  in 
the  NCTM  Standards.  Vygotsky's  theories  appear  particularly  useful  for  defining  and  guiding  the  changing  roles  of 
the  teacher  and  students. 

Vygotsky’s  Theories  of  Sociocultural  Learning 

Vygotsky's  sociocultural  theory  of  mind  serves  as  a middle  ground  between  formalism  and  constructivism 
(Kozulin,  1990;  Moll,  1990;  Vygotsky,  1986;  Wertsch,  1985;).  Concept  formation,  according  to  Vygotsky's  theory, 
is  a dynamic  interactive  action  between  the  concrete  and  the  abstract  that  docs  not  require  the  child  to  reinvent 
information  (as  does  constructivism),  nor  docs  it  expect  the  child  to  conceptualize  abstractions  without  having  first 
engaged  in  concrete  activities  that  support  the  formation  of  mental  models.  Vygotskian  theory  incorporates  several 
key  ideas  relative  to  levels  and  types  of  concept  formation,  the  role  of  collaborative  interchange  in  concept 
development,  and  instructional  configurations  that  guide  and  promote  concept  development. 

Language 

Language,  and  its  relationship  to  thinking,  is  at  the  foundation  of  Vygotsky's  sociocultural  theories;  he 
contended  that  language  is  an  indispensable  requisite  for  all  intellectual  growth.  Vygotsky  (1986)  contended  that  a 
concept  is  not  "...an  isolated,  ossified,  changeless  formation  but  an  active  part  of  the  intellectual  process,  constantly 
engaged  in  serving  communication,  understanding,  and  problem-solving."  (p.98)  He  argued  that  the  language  of 
adults  and  others  which  surrounds  the  child  is  a major  factor  in  the  child's  cultural  and  social  environment,  and  that 
this  language  influences  the  content  of  the  child's  thoughts  as  well  as  how  the  child  thinks.  Vygotsky  did  not  isolate 
verbal  interchanges  with  adults  as  a separate  component  in  the  process  of  a child's  concept  formation,  but  rather 
stressed  the  essential  interrelationship  between  sensory  material  and  words  as  mutual  components  of  a child's 
concept  development. 

The  child  and  the  adult  may  share  many  word  meanings,  but  Vygotsky  (1986)  cautioned  against  using  this 
as  evidence  that  the  mental  operations  of  the  child  and  adult  are  one  and  the  same.  He  maintained  that  it  is  through 
verbal  interactions  with  adults  and  sensory  interaction  with  their  culture  that  children  negotiate  meaning  and  form 
concepts.  Vygotsky  viewed  conceptual  thinking  as  being  purposeful  in  that  it  arose  in  answer  to  a problem: 

Memorizing  words  and  connecting  them  with  objects  docs  not  in  itself  lead  to  concept  formation;  for  the 

process  to  begin  a problem  must  arise  that  cannot  be  solved  otherwise  than  through  the  formation  of  new 

concepts,  (p.  100) 
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According  to  Vygotsky,  attainment  of  real  concepts  is  impossible  without  words  (Vygotsky,  1986; 
Vygotsky  & Luria,  1993).  Therefore,  he  considered  the  central  moment  in  concept  formation  to  be  the  point  at 
which  the  child  is  able  to  use  words  as  functional  tools.  He  considered  words  to  be  the  tools  or  means  that  direct 
mental  operations  and  focus  their  course  toward  the  solution  of  a problem. 

Spontaneous  and  Scientific  Concepts 

Vygotsky  recognized  the  need  to  connect  formal  concepts  learned  in  school  with  children's  understanding 
that  resulted  from  their  everyday  experiences  (Davydov,  1990;  Ratncr,  1991;  Schmittau,  1993).  The  crux  of  the 
Vygotskian  approach  to  instruction  lies  in  the  differentiation  Vygotsky  made  between  spontaneous  and  scientific 
concepts.  Spontaneous  concepts  are  formed  by  the  child  through  everyday  exposure  to  the  cultural  environment; 
they  are  unsystematic  and  highly  contextualized.  Scientific  concepts  are  mediated  through  formal  instruction;  are 
characterized  by  hierarchical,  logical  organization;  and  typically  arc  derived  from  the  highly  structured  activity  of 
classroom  instruction.  Spontaneous  and  scientific  concepts  are  characteristically  different,  yet  they  arc  not 
independent  of  each  odicr. 

While  Vygotsky  (1986)  asserted  that  scientific  and  spontaneous  concepts  develop  in  reverse  directions,  he 
maintained  that  they  are  closely  related,  with  each  supporting  the  formation  of  the  other.  He  perceived  spontaneous 
concepts  as  bottom-up  constructs  that  must  be  sufficiently  developed  before  the  child  is  able  to  absorb  a related 
scientific  concept.  In  Vygotsky’s  words: 

In  working  its  slow  way  upward,  an  everyday  concept  clears  a path  for  the  scientific  concept  and  its 
downward  development.  It  creates  a series  of  structures  necessary  for  the  evolution  of  a concept's  more 
primitive  elementary  aspects,  which  give  it  body  and  vitality.  Scientific  concepts,  in  turn,  supply  structures 
for  the  upward  development  of  the  child's  spontaneous  concepts  toward  consciousness  and  deliberate  use. 
(p.194) 

Zone  of  Proximal  Dcvclopmenl 

Vygotsky  (1986)  argued  against  traditional  cognitivist  theory,  which  suggests  that  a child's  developmental 
level  should  lead  instruction.  He  contended  that  the  child's  current  developmental  level  should  be  viewed  as  a 
starting  point  rather  than  as  an  ending  point;  a point  at  which  instruction  leads  rather  than  follows  development. 
Vygotsky  proposed  that  the  appropriate  level  of  instruction  lies  between  a child's  actual  mental  age  (i.e.,  his  or  her 
level  of  independent  functioning)  and  the  level  of  problem  solving  that  the  child  is  able  to  reach  with  assistance. 

Vygotsky  labeled  this  domain  the  zone  of  proximal  development  (ZPD).  With  adult  support,  the  child  is 
able  to  accomplish  more  than  he  or  she  is  able  to  accomplish  independently.  The  child's  state  of  development  is  the 
limiting  factor  in  this  process.  Mediation  leads  a child  through  his  ZPD  via  verbal  communication  and  collaborative 
interchange  with  an  adult  or  more  competent  peer.  As  the  child  becomes  more  capable  of  independent  thinking,  the 
child's  state  of  development  rises  and  mediation  can  reach  toward  a higher  instructional  level,  thus  leading 
development. 

The  child’s  movement  through  the  ZPD  is  characterized  by  concept  development  and  intellectual  movement 
which  arc  subject  to  the  child's  own  developmental  laws.  Movement  through  the  ZPD  involves  intelligent, 
conscious  imitation,  which  requires  that  the  child  first  understand  (he  field  structure  and  relationships  between 
objects.  This  conscious  imitation  should  not  be  confused  with  automatic  imitative  behavior  that  shows  no  signs  of 
conscious  understanding. 

Vygotsky's  theories  and  the  tenets  of  the  NCTM  standards  appear  to  have  many  areas  of  commonality. 
Both  imply  multidimensional  thinking  rather  than  one-dimensional  learning.  Both  contend  that  intelligent, 
conscious  imitative  beh  \ior,  rather  than  automatic  imitative  behavior,  leads  to  concept  development.  Vygotsky's 
theories  and  the  NCTM  v tandards  agree  on  the  value  of  cultural  contcxtuali/.ation  in  the  negotiation  of  meaning  and 
reasoning  skills,  and  on  (he  importance  of  collaboration,  communication  and  language  in  the  process  of  concept 
development.  Thus,  it  appears  that  Vygotsky's  theories  may  offer  appropriate  and  useful  guidelines  identifying  the 
characteristics  of  effective  uses  of  various  technologies  in  the  mathematics  curriculum. 
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The  Role  of  Technology  in  the  Mathematics  Curriculum 

The  Standards  are  purposefully  general  about  specific  types  of  technology  and  their  uses.  In  doing  so,  ihey 
recognize  the  dynamic  state  of  technological  advancement  and  assume  that  ongoing  change  in  technology  will 
continue  to  impact  the  nature  of  mathematics,  The  focus  of  the  standards  is  on  the  goals  and  objectives  of  the 
mathematics  curriculum,  relegating  technology  to  the  role  of  a facilitating  tool  for  concept  formation.  It,  therefore, 
seems  appropriate  to  explore  available  technologies  and  to  investigate  their  potential  supportive  application  in  the 
mathematics  curriculum.  Issues  related  to  technology's  role  in  mathematics  education  include  content 
contextualization  in  the  negotiation  of  meaning  and  reasoning  skills,  collaboration,  communication  and  language  in 
the  process  of  concept  development. 

Mindtools 

Jonasscn  (1996)  discussed  using  "mindtools,"  which  arc  computer-based  tools  and  learning  environments 
that  have  been  "...adapted  or  developed  to  function  as  intellectual  partners  with  the  learner  in  order  to  engage  and 
facilitate  critical  thinking  and  higher-order  learning"  (d.9).  The  functional  role  of  mindtools  is  twofold:  (a)  to  extend 
the  learner’s  cognitive  functioning  during  the  learning  process;,  and  (b)  to  engage  ihe  learner  in  operations  while 
constructing  knowledge  that  they  would  not  have  been  able  to  accomplish  otherwise.  According  to  Jonasscn, 
mindtools  support  a learning  environment  in.  whUh  students  arc  a!  ;e  to  process  information  intentionally  and 
meaningfully,  to  build  on  prior  learning,  to  elaborate  on  new  knowledge,  to  interrelate  new  knowledge  with  their 
prior  knowledge,  and  to  consciously  reflect  on  their  learning.  He  further  suggested  that  mindtools  might  be 
enhanced  through  collaborative  and  cooperative  efforts  between  and  among  students  and  teachers  in  the  learning 
community.  Mindtools,  therefore,  appear  to  offer  a supportive  structure  to  a Vygotskian  approach  to  mathematics 
learning.  Among  the  applications  Jonasscn  lists  as  mindtools  are  databases,  spreadsheets,  and  computer  mediated 
communication.  Each  of  these  will  be  discussed  in  the  following  sections. 

Databases 

Databases  are  computerized  record-keeping  applications  which  allow  easy  entry  and  fast,  flexible  retrieval 
of  information.  The  two  basic  components  of  a database  arc  the  field  entries,  which  arc  the  individual  categories 
under  which  data  is  entered,  and  the  record,  which  is  a compilation  of  the  representative  fields.  The  database  is  a 
collection  of  records  within  which  like  categories  of  information  arc  stored.  This  is  similar  to  having  a collection  of 
index  cards  on  which  one  would  store  the  names,  addresses,  and  phone  numbers  of  all  friends  and  family.  The 
collection  of  index  cards  would  represent  the  database;  each  index  card  is  equivalent  to  a record,  while  each  picc~  of 
information  on  the  card  equates  to  a field.  Electronic  databases  are  much  more  flexible  than  a system  based  on 
index  cards,  since  the  records  may  be  sorted  and  organized  by  any  part  of  the  information  in  separate  fields.  This 
allows  quick  retrieval  of  any  particular  kind  of  information  the  student  may  wish  to  access. 

Construction  of  databases  requires  that  learners  identify  and  classify  the  important  underlying  constructs  of 
their  information  and  the  contextual  relationships  within  the  content.  Students  must  analyze  and  comprehend  the 
information  they  wish  to  store.  A database  management  system  can  help  students  organize,  interrelate  and 
synthesize  content  information  in  a more  meaningful  manner.  Using  databases  extends  critical,  creative  and 
complex  thinking  skills  (Jonasscn,  1996). 

Classification  involves  forming  an  organizational  schema  among  elements  in  which  noticing  similarities 
and  differences  of  objects  is  foundational  (Sovichik,  1996).  The  ability  to  classify  and  to  sort  is  fundamental  in  the 
hierarchy  of  mathematical  thinking  skills  (Holmes,  1995;  Kennedy,  & Tipps,1996;  Troutman  & Lichtcnberg,  1995). 
Hallmark  to  Vygotsky’s  theory  of  concept  development  is  the  child’s  movement  from  the  ability  to  form 
relationships  between  and  among  physical  objects  on  various  levels  to  the  ability  to  abstract  properties  independent 
of  the  physical  object  (Vygotsky,  1986).  According  lo  Vygotsky,  the  child's  classifications  progress  from  (a)  an 
initial  level  of  forming  concrete  and  factual  bonds  which  are  based  on  external  likeness,  to  (b)  grouping  by  contrast 
where  objects  are  categorized  on  the  basis  of  a single  trail  on  which  the  objects  differ  and  thus  complement  one 
another,  to  (c)  linking  meanings  in  a chain-likc  manner  where  meaning  is  carried  over  from  one  link  to  the  next,  and 
finally  to  (d)  fluid  connections  between  and  among  objects. 

Databases  allow  visual  representation  and  provide  a dynamic  repository  for  information  that  may  support 
the  child  from  the  most  elementary  through  the  most  complex  stages  of  classification  and  abstractions.  Further,  the 
construction,  maintenance  and  use  of  databases  encourage  collaborative  enterprises. 

In  a school  setting,  students  could  use  databases  with  increasing  complexity  from  the  early  grades 
throughout  their  secondary  education.  Databases  can  be  used  to  reflect  authentic  information  gathered  by  the 
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students,  and  to  provide  a focus  for  the  analysis  and  interpretation  of  the  information  stored.  For  younger  students, 
this  information  may  revolve  around  personal  and  concrete  information  (c.g.,  tracking  weather  information  hy  day) 
while  for  older  students  it  may  contain  data  relative  to  more  complex  and  more  abstract  subject  matter  (c.g., 
categorizing  mathematical  functions). 

Spreadsheets 

Electronic  spreadsheets  arc  a tool  for  organizing  and  manipulating  numerical  data;  therefore  they  can  help 
students  manage  numerical  information  and  relationships.  The  spreadsheet  is  a two-dimensional  matrix  of  cells. 
Columns  are  identified  by  letters,  and  rows  are  identified  by  numbers.  Each  cell  may  contain  text,  a numerical 
formula,  a value  or  a function.  Spreadsheets  have  three  main  functions  that  make  them  particularly  conducive  to  the 
integration  and  extension  of  the  mathematics  curriculum:  (a)  storing  information,  (b)  calculating  functions  which 
relate  information,  and  (c)  presenting  the  results  of  these  calculations  as  well  as  the  underlying  information  (Grabc, 
& Grabc,  1998;  Jonassen,  1996;  Roblycr,  Edwards,  & Havriluk,  1997).  Many  spreadsheets  have  the  additional 
capability  of  easily  converting  numerical  information  into  various  graphical  representations. 

At  the  most  fundamental  level,  developing  and  using  a spreadsheet  as  a mindtool  requires  students  to 
identify  rules  and  to  understand  relationships.  On  more  advanced  levels,  students  generate  new  rules  that  describe 
and  organize  relationships  (Jonassen,  1996).  Jonassen  cautioned,  however,  that  if  students  merely  engage  in  data 
entry  they  are  engaging  in  lower  level  learning  on  the  knowledge  and  comprehension  levels.  On  the  other  hand, 
answering  complex  questions  elevates  learning  to  the  application  level.  Students  who  are  identifying,  describing  and 
classifying  relationships  in  terms  of  higher  order  rules  are  participating  in  higher  levels  of  critical  thinking. 

Vygotsky  (1986)  identified  stages  of  abstraction  that  coincided  with  the  various  levels  of  classification 
abilities  during  the  process  of  concept  formation.  At  the  most  elementary  level  of  concept  development,  Vygotsky 
maintained  that  the  child  is  able  to  abstract  whole  groups  of  properties  based  on  functional  similarities  (e.g.,  spoon, 
plate,  cup).  The  child  then  progresses  to  abstracting  a single  property  (c.g.,  cup  and  glass  for  drinking),  and 
eventually  to  abstracting  physical  properties,  impressions,  and  meaning  or  other  attributes  not  tied  to  physical 
attributes.  Vygotsky’s  contention  supports  Bruner’s  (1960)  advocacy  of  teaching  mathematical  concepts  on  a 
continuum  that  progresses  from  the  enactive  (concrete  representations)  through  the  ikonic  (pictorial  and  graphical 
representations)  and  finally  to  the  symbolic  (characterized  by  the  ability  to  use  words  and  other  symbols  as 
abstractions). 

By  nature,  spreadsheets  arc  much  more  abstract  than  databases;  their  appropriate  use  is  limited  to 
quantitative  problem  solving  (Grabc,  & Grabc,  1998;  Jonassen,  1996;  Newby,  Stcpich,  Lehman,  & Russell,  1996). 
A spreadsheet,  when  appropriately  integrated  into  instruction,  can  be  a higher  level  organizational  and  functional 
tool  that  may  augment  and  support  the  child’s  conceptual  progression  through  his  or  her  ZPD.  As  with  databases, 
spreadsheets  may  also  promote  increased  collaboration  and  communication  among  students. 

A classroom  teacher  may  effectively  and  flexibly  use  a spreadsheet  by  structuring  the  content  to 
appropriately  reflect  the  students’  ZPDs.  For  example,  younger  students  could  use  more  concrete  experiences  as 
data  sources  (c.g.,  by  preparing  different  soap  solutions  and  collecting  data  on  the  bubbles  produced)  while  more 
advanced  students  might  use  more  sophisticated  data  (c.g.,  survey  information)  from  which  they  may  draw  abstract 
inferences.  Used  in  this  manner,  spreadsheets  offer  promise  both  as  an  instructional  medium  and  as  an  alternative 
assessment  tool. 

Computer  Mediated  Communication 

Computer  mediated  communication  (CMC)  mediates  communication  between  and  among  individuals  and 
groups  of  individuals.  Through  computer  technology  and  networks,  individuals  who  may  be  as  close  as  an  adjacent 
room  or  as  geographically  distant  as  in  another  state  or  country  may  readily  communicate  either  synchronously  or 
asynchronously.  Synchronous  communication  requires  that  participants  communicate  with  each  other  at  the  same 
time  and  requires  that  connections  be  established  for  both  receiving  and  transmitting  information.  Presently 
synchronous  CMC  requires  more  sophisticated  linkages  and  is  more  expensive  than  asynchronous  CMC.  For  these 
reasons,  access  to  synchronous  CMC  is  more  limited  in  school  settings. 

Asynchronous  communication  docs  not  require  that  those  communicating  be  connected  at  the  same  time. 
Because  the  linkages  arc  simplified  and  less  expensive,  it  is  more  commonly  found  in  school  settings.  Asynchronous 
communication  may  take  one  of  three  forms:  (a)  onc-to-onc,  as  in  e-mail  or  use  of  search  engines;  (b)  onc-to-many, 
as  in  bulletin  boards;  or  (c)  many-to-many,  as  in  conferencing  or  listscrvs.  All  of  these  uses  permit  communications 
to  transcend  time  and  space  limitations. 


Bednar,  Cunningham,  Duffy,  and  Perry  (1992)  contended  that  identifying  and  applying  a particular 
learning  theory  to  instructional  design  is  crueial  when  using  CMC.  Khan  (1997)  developed  a list  of  attributes  that 
characterize  CMC  implemented  over  the  World  Wide  Web.  Many  of  these  characteristics  would  lend  themselves  to 
meaningful  inclusion  in  designing  effective  instruction  that  is  consonant  with  Vygotsky's  learning  theories  and  with 
the  NCTM  Standards.  Among  those  attributes  that  relate  to  instruction  that  is  consistent  with  both  Vygotsky  and  the 
Standards  arc:  (a) 

1.  interactivity,  which  allows  students  to  interact  with  teachers,  peers,  and  online  resources, 
and  when  asynchronous,  also  allows  time  for  reflective  learning  and  self  pacing; 

2.  multimedia,  which  extends  the  potential  to  address  all  learning  styles  through  the 
incorporation  of  various  media  including  text,  video,  animation,  audio,  graphics  and  simulations; 

3.  open  systems,  which  permit  Ihc  learner  to  move  outside  of  his  or  her  immediate 
environment  and  to  experience  other  resources,  perspectives,  and  cultures; 

4.  distributed  resources,  which  make  available  relevant  information,  a variety  of  materials, 
and  easily  attainable  multimedia  representations  of  a wide  range  of  subject  matter; 

5.  collaborative  learning,  which  emphasizes  cooperative  efforts  and  communication,  and 
which  engages  students  in  the  negotiation  and  validation  of  meaning; 

6.  formal  and  informal  environments,  which  support  instruction  on  a continuum  from 
instructor  driven  to  student-centered  discovery;  and 

7.  authentic  resources,  which  give  students  access  to  real  world  problems  and  to  real  world 
data. 

Bannan  and  Milheim  (1997)  discussed  general  instructional  methods  that  are  useful  with  web-based 
instruction.  The  particular  methods  they  presented  take  advantage  of  the  characteristics  listed  by  Khan,  and  also  arc 
consonant  with  a Vygotskian  approach  to  instruction.  Particular  methodologies  Bannan  and  Milheim  described  that 
directly  support  Vygotsky’s  theories  include:  (a)  facilitation,  (b)  inside  and  outside  collaboration,  (c)  apprenticeship, 
(d)  content  generation,  (e)  role-playing,  and  (0  modeling.  The  authors  went  on  to  discuss  general  activities  that 
could  be  adapted  to  support  group  communication,  course  interaction,  discussion,  and  facilitation.  All  of  these  arc 
applicable  to  mathematics  learning. 

Asynchronous  communication  has  given  rise  to  more  than  just  ideas;  it  has  enabled  a plethora  of  actual 
classroom  projects  sponsored  by  organizations  such  as  the  Eisenhower  Clearinghouse,  NASA,  and  PBS.  Several  of 
these  will  be  described  in  the  following  section. 

In  1996  and  1997,  among  the  various  virtual  field  trips  offered  by  NASA,  classrooms  could  enroll  in  a 
virtual  expedition  to  Antarctica.  While  an  actual  scientific  expedition  was  on  site  in  Antarctica,  students  in  the 
United  Slates  communicated  via  e-mail  with  the  scientists  in  Antarctica  and  with  other  classes  involved  in  the 
project.  During  the  project,  the  scientists  e-mailed  to  participating  students  not  only  their  personal  journals  but  also 
reports  on  activities,  weather  conditions,  and  problems  they  encountered.  Students  could  also  access  these  updates 
on  a NASA-sponsored  World  Wide  Web.  The  web  site  included  relevant  project  information  on  scientific  projects 
in  graphics,  audio,  and  video  formats.  The  project  provided  a contextualized  background  for  extension  activities 
that  gave  students  an  opportunity  to  extend  mathematical  concepts  by  manipulating  and  interpreting  data  on  various 
levels. 

Additional  asynchronous  communications  sites  sponsored  by  diverse  organizations,  such  as  Scholastic, 
Family  Math,  and  various  colleges  of  education  arc  available  on  the  Internet  or  the  World  Wide  Web.  These  sites 
engage  students  in  on-line  problem  solving  projects  and  in  other  math-related  activities.  They  permit  students  to 
access  databases  of  relevant  information.  Students  may  share  their  ideas  and  respond  to  others  participants’  ideas 
about  particular  questions  or  topics  by  subscribing  to  and  participating  in  a listserv.  A listserv  is  a common  mailbox 
that  receives  mail  and  then  distributes  that  mail  to  all  subscribers.  Topical  sessions  may  be  facilitated  and 
moderated  by  an  expert  who  guides  discussions  by  providing  pertinent  questions.  In  this  venue  students  can 
participate  in  collaborative  processes  to  negotiate  meaning,  to  validate  methods,  to  resolve  misconceptions,  and  to 
form  and  extend  concepts. 

As  access  to  asynchronous  CMC  becomes  more  widespread  among  school  communities,  possibilities  for 
supporting  connectivity  increase.  Partnerships  between  classrooms  could  be  formed  with  children  in  geographically 
distant  schools.  The  children  could  share  information  about  their  communities,  and  compare  data  from  their  partner 
schools  with  their  local  data.  Students  could  organize  these  data  and  develop  databases  and  spreadsheets. 
Analyzing  data  from  various  locations  could  provide  a basis  for  forming  inferences  about  topics  such  as  topography, 
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industry,  and  population.  Students  would  then  be  able  to  share  their  analyses  with  partner  schools  in  order  to 
validate  and/or  correct  their  inferences. 

Conclusion 

Vygotsky's  sociocultural  theories  appear  to  support  the  change  in  direction  of  mathematics  education  that  is 
advocated  by  the  NCTM  Standards.  The  standards  call  for  students  to  become  literate  in  a mathematical  language 
through  which  they  can  articulate,  communicate  and  solve  problems.  The  change  in  instructional  focus  from  a 
product  to  a process  orientation  finds  affirmation  in  Vygotsky's  theories  of  concept  development,  and  the  change  in 
the  roles  of  both  teacher  and  student  are  consonant  with  Vygotsky's  views  of  mediation  and  the  ZPD. 

Appropriately  implemented,  technology  offers  tools  to  the  classroom  that  may  promote  high  level  thinking 
skills  and  support  concept  development.  Asynchronous  computer  mediated  communication  allows  students  access 
to  other  individuals,  information,  and  experiences  that  were  previously  unavailable  in  the  classroom  setting.  CMC 
may  be  implemented  in  a number  of  ways  to  support  the  mathematics  curriculum  through  increased  collaboration 
and  mediation. 

As  access  to  CMC  becomes  more  universal,  new  ways  of  assuring  supportive  connectivity  between  and 
among  classroom  communities  should  be  explored.  Through  technology,  the  perceived  and  functional  distance 
between  learning  communities  is  becoming  smaller.  This  new  potential  for  communicating  is  important  not  only  for 
students,  but  also  for  the  educators  who  implement  the  curriculum.  As  we  open  the  confines  of  time  and  space  and 
virtually  remove  the  walls  that  isolate  the  classroom,  we  will  move  into  unexplored  ways  of  communicating, 
learning,  and  teaching  with  technology.  In  this  process,  wc  must  keep  in  mind  that  the  nature  of  the  child  docs  not 
change  and  that  our  efforts  to  educate  should  be  consonant  with  a sound  theoretical  base. 
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Design  Model  for  Learner-Centered,  Computer-Based  Simulations'2 

Chandra  L.  Hawley 
Thomas  M.  Duffy 
Indiana  University 

A simulation  has  been  described  as  “an  event  in  which  the  participants  have  (functional)  roles,  duties  and 
sufficient  key  information  about  the  problems  to  carry  out  these  duties  without  play  acting  or  inventing  key  facts. 
[Participants]  keep  their  own  personalities  but  take  on  a job,  duties,  responsibilities  and  do  the  best  they  can  in  the 
situation  in  which  they  find  themselves.”  (Jones,  1995)p.  18.  According  to  this  definition,  the  simulation  is  an  ideal 
Icarncr-ccntcred  tool  for  use  in  the  classroom.  After  all,  they  place  the  learner  in  a central  decision  making  role  and 
allow  her  to  encounter  problems  in  a “real  world”  environment.  In  fact,  one  could  claim  that  the  simulation  is  a 
cognitive  apprenticeship  in  an  authentic  environment.  Simulations  can  be  important  tools  for  enhancing  school 
learning  environments. 

Considering  this  useful  application  of  simulations  and  the  affordability  and  relative  ease  of  development, 
why  are  we  not  seeing  more  workplace  simulations  developed  for  the  schools?  While  simulation  environments  arc 
extremely  popular  in  industry  and  effective  in  job  training  (e.g.,  Schlecter,  Bessemer,  & Kolosh,  1992;  Towne, 
1995)  few,  if  any,  of  these  workplace  training  systems  are  transferable  to  the  School  setting.  There  are  several 
reasons  for  this.  At  a practical  level,  the  technology  is  typically  too  advanced  and  expensive  to  be  feasible  for 
schools  to  own.  From  a pedagogical  point  of  view,  the  focus  on  job  training  will  typically  result  in  the  performance 
requirements  being  too  complex  for  use  by  children  in  school.  Finally,  from  a curriculum  perspective,  most  schools 
would  reject  a focus  on  direct  job  training  with  the  goal  being  to  prepare  a person  for  a particular  job.  The  role  of  K~ 
12  education  is  to  prepare  students  to  be  active,  competent,  and  informed  citizens  in  our  society,  not  specific  job 
training. 

In  order  to  change  this  trend  and  provide  students  with  these  potentially  useful  apprenticeship  opportunities 
in  either  cognitive  or,  in  a few  cases,  manual  skill  training,  there  are  certain  considerations  which  must  be  looked  at. 
In  order  to  realistically  find  a home  for  workplace  simulations  in  the  schools,  we  must  look  for  simulations  that  use 
affordable  technology,  have  performance  demands  that  are  feasible  for  K-12  students,  and  focus  on  introducing  the 
learner  to  the  culture  of  professions,  Lc.t  introducing  them  to  the  basic  skill  requirements,  the  basic  concepts,  and  the 
problems  one  encounters  in  the  profession.  In  our  search  for  workplace  - or  citizenship  related  - simulations  of  this 
type  the  primary  examples  we  found  were  the  work  of  Classroom  Inc.  and  the  work  at  the  Institute  for  the  Learning 
Sciences  (ILS)  (Schank,  Fano,  Jona,  and  Bell,  1993;  Ricsbcck,  1996).  There  are  very  few  other  examples.  Overall, 
we  were  very  surprised  by  the  small  numbers  since  these  simulations  seem  to  hold  such  potential  within  the  learning 
theory  framework  of  situated  learning  and  considering  that  industry  has  found  simulations  to  be  particularly 
effective  in  job  training  (sec,  e.g.,  Towne,  1995). 

It  quickly  became  apparent  in  our  search  for  computer-based  simulation  environments  for  schools  that  there 
is  a shortage  of  these  kinds  of  environments.  Wc  also  quickly  realized  that  there  is  little  guidance  for  effective 
design  of  simulations  of  this  kind.  The  research  that  docs  exist  lends  to  take  an  information  processing  perspective 
emphasizing  information  transmission  through  practice  and  feedback  (Reigeluth  & Schwartz,  1989).  While  this  kind 
of  approach  may  be  appropriate  for  simulations  that  teach  particular  information  or  skills,  such  as  flight  simulators, 
this  approach  may  be  sufficient.  However,  wc  would  like  to  examine  how  one  might  design  a simulation  within  a 
situated  or  constructivist  framework.  After  we  have  examined  the  various  aspects  of  this  model,  wc  will  demonstrate 
its  use  by  using  it  to  examine  one  of  the  Classroom  Inc.  Simulations  we  have  worked  with,  The  Chelsea  Hank. 


2 The  research  reported  here  V'as  supported  in  pan  by  a grant  from  the  Andrew  W.  Mellon  Foundation  and  the 
Russell  Sage  Foundation. 
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Design  Criteria 

So  what  docs  a situated  learning  or  constructivist  simulation  look  like?  There  are  many  sources  we  can  go 
to  for  criteria  to  answer  this  question.  For  the  purposes  of  our  model,  we  look,  to  two  sources:  the  work  of  Savery 
and  Duffy  (Saver)'  & Daffy,  1996)  and  the  American  Psychological  Association’s  “Learner-Centered  Psychological 
Principles.”  The  view  of  learning  we  are  taking,  as  already  stated,  is  a constructivist  one  characterized  by  a set  of 
philosophical  views  that  include  the  idea  that  we  learn  through  our  interactions  with  the  environment;  puzzlement  is 
not  only  the  stimulation  for  learning,  but  also  affects  our  organization  of  what  we  have  learned;  and  knowledge 
evolves  through  both  social  negotiation  and  through  each  person  evaluating  the  viability  of  their  understandings. 
This  basic  philosophical  basis  leads  to  many  exciting  learning  opportunities  including  those  that  simulations  can 
facilitate. 

The  five  primary  criteria  for  the  development  of  simulations  we  propose  include; 

• The  problem  needs  to  be  authentic. 

Problems,  goals,  and  obstacles  need  to  be  consistent  with  the  environment  which  is  being  simulated  so  that 
the  learner  perceives  both  the  environment  and  problems  to  be  authentic.  By  maintaining  this  authenticity 
rather  than  artificially  simplifying  the  task  the  student  is  working  on,  motivation  stays  higher  and  learners 
will  be  more  likely  to  stay  on  task  without  coercion  (APA,  November,  1997). 

• The  cognitive  demand  in  learning  are  authentic. 

In  these  kinds  of  simulation  environments,  the  learner  becomes  the  owner  of  their  own  decision  making 
process.  They  get  to  choose  how  to  go  about  finding  the  best  solution  they  can.  Again,  this  ..elps  keep  the 
intrinsic  motivation  level  high.  Further,  the  information  gathered  during  engagement  in  the  simulation  is 
not  artificially  provided.  Therefore,  the  student  is  experiencing  what  working  in  a particular  environment  is 
really  like  only  without  the  high  stakes  associated  with  true  workplaces. 

• Scaffolding  supports  a focused  effort  relevant  to  the  learning  goals. 

Simulation  environments  arc  ideal  tools  for  this  kind  of  scaffolding.  They  can  remove  or  lessen  certain 
constraints  that  would  get  in  the  way  of  the  learning  process  in  the  real-life  counterpart  of  the  simulation. 
For  instance,  in  the  ease  of  the  simulation  we  will  examine  in  this  paper,  the  student  who  is  working  as  a 
bank  teller  docs  not  need  to  worry  about  time  eonsidcrations  or  about  how  many  more  people  arc  wailing  in 
the  line  behind  the  current  customer.  These  kinds  of  real-life  constraints  would  only  gel  in  the  way  of  the 
students  learning  how  to  deal  with  ihc  problem-solving  tasks  they  arc  being  faced  with.  It  is  important  to 
. point  out  that  scaffolding  should  not  be  confused  with  simplifying  the  conditions  in  a non-aulhentic  way.  In 
our  simulation,  students  still  work  with  individual  customers  on  real-life  banking  transaction,  however, 
their  work  situation  has  certain  elements  removed  from  it  that  would  only  serve  as  distractors  in  the 
learning  process. 

• Coaching  promotes  learning  rather  than  directing  or  correcting  performance. 

In  our  work  with  the  various  simulation  environments  (Hawley,  1997;  Hawley  & Duffy,  1997;  Duffy  et  ah, 
19%) we  have  worked  w;th,  this  has  proven  to  be  a vital  point.  The  role  of  the  teacher  in  one  of  these 
environments  should  be  one  of  a coach  - a person  who  asks  students  questions  on  the  cutting  edge  of  their 
knowledge,  pushes  them  to  be  reflective  and  thoughtful  in  their  decision  making,  and  enables  the  students 
to  become  independent  problem  solvers.  As  von  Glaserfcld  (1993)  pointed  out,  “It  is  crucial  for  the  teacher 
to  get  some  idea  of  where  they  [the  students]  are  (What  concepts  they  seem  to  have  and  how  to  relate 
them)"  (p.  192).  Savery  and  Duffy  (1996)  add  to  this  saying  that  “It  is  essential  that  the  teacher  value  as 
well  as  challenge  the  learner’s  thinking.  The  teacher  must  not  take  over  the  thinking  for  the  learner  by 
telling  the  learner  what  to  do  or  how  to  think,  but  rather  teaching  should  be  done  by  inquiring  at  the 
“leading  edge”  of  the  learner’s  thinking”  (p.  139). 

In  our  work  to  date,  we  have  found  a much  higher  instance  of  teachers  cither  directing  students  by  giving 
them  explicit  directions  for  how  tc  play  the  simulation  or  standing  back  and  letting  the  student  work 
entirely  independently  with  no  guidance,  reflective,  or  mctacognitive  activity.  Our  belief  based  on 
observations  of  many  classrooms  where  the  simulations  were  being  used  is  that  the  students  who  arc  left 
alone  or  those  who  are  directed  through  the  simulation  miss  out  on  a large  portion  of  the  learning.  As  with 
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any  other  mechanical  tasks  that  students  withstand,  the  simulation  becomes  something  to  work  through  just 
to  get  a grade.  As  the  APA  points  out  (APA,  November,  1997),  there  arc  many  different  ways  a successful 
learner  can  create  and  use  a variety  of  reasoning  strategics.  A teacher  who  is  acting  as  a coach  allows  the 
students  to  explore  the  various  ways  of  thinking  about  a problem,  possibly  through  questioning  students 
about  their  strategics,  thinking  aloud  about  her  own  strategy,  or  having  students  share  the  different 
approaches  they  took  to  solving  the  problem  with  their  classmates.  It  is  this  kind  of  teaching  style  that  is 
necessary  in  order  for  the  computer-based  workplace  simulations  to  be  most  effective. 

• Reflection  supports  abstracting,  synthesizing,  and  extending  the  learning, 

“Doing”  the  simulation  is  accompanied  by  “reflecting”  about  what  happened  during  the  simulation.  It  is  in 
this  reflection  process  that  learners  can  begin  10  synthesize  what  they  have  just  done.  The  reflection 
activities  can  take  place  as  class  discussions,  small  group  discussions,  or  journal  writing  assignments. 
Special  emphasis  might  be  placed  on  the  discussion  approaches  to  reflection  considering  the  social 
negotiation  aspects  of  learning  - we  learn  by  bouncing  our  ideas  off  of  other  people  who  may  have 
different  ideas  than  our  own. 

• The  environment  should  be  engaging. 

While  everything  listed  so  far  should  lead  to  an  engaging  environment,  special  attention  should  be  paid  to 
this  aspect.  If  the  students  buy  into  the  simulation,  they  will  be  more  likely  to  remain  engaged  and 
maximize  their  learning  experience. 

With  these  criteria  as  a basis,  we  will  now  explore  The  Chelsea  Bank.  Keep  in  mind  that  the  ultimate  goal 
of  this  simulation  and  other  simulations  of  this  type,  is  the  development  of  problem-solving  skills  in  the  students 
who  use  it.  This  simulation  goes  a step  further  in  that  it  requires  not  only  good  problem  solving,  but  also  puts  some 
emphasis  on  ethical  problem  solving  and  issues  of  customer  service  associated  with  the  everyday  work  world. 

The  Chelsea  Bank 

An  Overview  of  the  Simulation  Package. 

The  Chelsea  Bank  package  is  more  than  a piece  of  software.  It  includes  the  software,  a teacher’s  guide,  an 
audio  tutorial  workbook,  and  extra  software  focusing  on  savings  and  checking  accounts.  As  with  all  of  the 
Classroom  Inc.  Software,  the  program  can  only  be  used  by  teachers  who  have  gone  through  the  Classroom  Inc. 
training  program  which  focuses  on  not  only  the  technological  aspects  of  integraling  software  into  the  classroom,  but 
also  using  the  software  in  a way  that  is  consistent  with  the  guidelines  we  have  outlined.  Classroom  inc.  Is  extremely 
concerned  about  the  implementation  of  Ihcir  programs  in  the  classroom  and  has  spent  several  years  working  on  their 
training  programs.  Beyond  the  training  and  wrilten  materials,  adoption  of  the  software  also  ensures  the  teacher  a 
support  network.  Teachers  using  the  simulation  get  help  through  site  visits  and  through  communications  with  the 
Classroom  Inc.  offices. 

Central  to  the  teaching  strategy  set  forth  by  Classroom  Inc.,  the  students  work  on  the  program  in  groups  of 
three  to  four  students.  This  is  done  in  an  effort  to  help  the  students  learn  to  work  in  a team.  It  is  also  consistent  with 
the  idea  that  “Learning  is  influenced  by  social  interactions,  interpersonal  relations,  and  communication  with  others” 
(APA,  November,  1997). 

Also  central  to  the  Classroom  Inc.  philosophy,  activities  should  be  done  before  and  after  the  actual 
scenarios  in  order  to  help  students  tie  the  computer-based  activities  to  real-life  activities.  In  some  cases,  these 
activities  arc  transfer  ones  in  which  the  students  have  to  talk  or  write  about  things  that  have  happened  in  their  own 
lives  that  parallel  situation  in  the  scenarios.  Other  activities  have  the  students  build  on  the  world  presented  to  them 
in  the  computer  by  doing  things  such  as  taking  a ficldtrip  to  the  bank,  running  a mock  economy,  or  designing  an  ad 
to  hire  a new  teller. 

An  Overview  of  the  Simulation. 

The  Chelsea  Bank  consists  of  15  scenarios  in  which  the  groups  of  learners  act  as  cither  a bank  teller  (8 
scenarios)  or  a customer  service  representative  (7  scenarios).  The  learner  is  asked  to  do  standard  banking  activities 
including  cash  a check,  deposit  a check  oi  cash,  open  a loan  or  credit  card,  handle  customer  complaints,  or  open  a 
checking  account. 
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As  an  illustration  of  how  the  simulation  works,  we  will  consider  one  of  the  teller  situations.  Each  scenario 
starts  on  a new  day.  The  screen  show  the  student  the  view  from  behind  the  teller  desk.  The  students  can  see  the 
lobby  of  the  bank.  Once  they  click  the  ‘‘Next”  sign,  the  students  read  a short  note  about  who  the  new  customer  is  and 
what  he  or  she  would  like.  Often  the  information  given  in  this  introduction  includes  personality  information,  e.g., 
your  best  friend  walks  up  to  your  window  and  is  in  a hurry  or  a grumpy  customer  approaches  your  window.  Then,  a 
customer  walks  up  to  the  window  and  places  his  or  her  transaction  under  the  window  and  says  (via  text  box)  what  he 
or  she  wants.  At  this  point,  the  students  must  act  as  the  teller  and  decide  what  to  do.  They  must  decide  if  and  bow 
the  transaction  will  be  completed. 


Hello.  It's  so  nice  to  see 
you  again,  You  tmurtbe 
glad  it'f  Friday.  You  *11 
wozk  *>  hard  her*, 

I*d  like  to  deposit  time 
four 


4»9»8k>9) 


A,  Acceptable  ID. 


• One  of  the  following 
pieces  of  I D.: 

• Signature  card  for  an 
account  at  this  bank 

• This  bank's  ATM  card 


P*gp8 

•OR  2 piece®  of  personal 
I.D.: 

• Driver's  license 

• Employee  ID.  card 
with  signature  and 
photo 

• CHELSEA  credit  card 
with  signature  and 
embossed  name 

• Passport  with 
signature  and  photo 


Figure  2.  Manual  page  from  The  Chelsea  Bant 
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In  order  to  facilitate  the  decision-making  process,  the  program  includes  a variety  of  resources  that  the 
students  can  use,  if  they  choose,  in  order  to  make  a decision  including” 
the  bank  policy  and  procedure  manual 
customer  accounts  database 
credit  report  database 
the  materials  submitted  by  the  customer 

multiple  points  of  view  (when  dealing  with  customer  complaints) 
application  forms  when  they  are  appro'  date 

Once  the  learners  make  a decision,  they  go  to  a “decision”  screen  where  they  are  presented  four  or  five 
alternative  decisions  and  they  must  choose  one.  These  decisions  arc  always  available  during  the  scenario,  meaning 
that  the  students  can  make  a decision  with  no  evidence  to  support  it,  thereby  bypassing  the  problem-solving  process 
entirely.  They  could  also  choose  to  use  a process  of  deduction  to  solve  the  problem  by  looking  at  the  choices  early  in 
the  decision-making  process,  then  disproving  them  to  find  the  “right”  answer. 

Once  a choice  is  made,  the  students  are  asked  to  reflect  on  their  decision.  They  are  asked  to  describe  the 
rule  of  good  banking  they  used  to  solve  the  problem,  as  well  as  what  the  results  to  themselves,  the  customer,  and  the 
bank  will  be  because  of  their  decision.  The  program  provides  a space  for  writing  that  can  be  printed  out,  but  without 
teacher  guidance,  there  is  no  guarantee  that  the  students  will  answer  the  questions. 

Once  the  reflection  questions  are  answered,  the  students,  have  an  opportunity  to  read  about  the 
consequences  of  their  decision.  They  can  also  read  the  consequences  of  the  choices  they  did  not  pick. 

Evaluation  of  The  Chelsea  Bank 

To  answer  our  question  about  The  Chelsea  Bank  - to  what  degree  does  the  design  of  the  program  support 
the  development  of  problem  solving  skills  in  completing  banking  transactions,  in  interacting  with  customers,  and  in 
making  ethical  decisions  - we  will  evaluate  it  using  the  design  criteria  set  forth  in  this  paper. 

Is  the  problem  authentic?  The  problems  set  forth  in  The  Chelsea  Bank  arc  certainly  authentic.  This  is  one  of  the 
strengths  of  the  program.  The  authenticity  of  presentation  includes  the  types  of  transactions  presented  and  the  kinds 
of  customers  that  are  dealt  with.  A further  strength  of  Chelsea  is  that  the  students  actually  perceive  the  situation  as 
authentic.  In  our  interviews  with  students  at  half  a dozen  schools,  the  students  consistently  commented  on  how  real 
the  simulation  was  - that  it  was  like  really  being  at  the  bank.Furthermore,  they  described  the  scenarios  by  talking 
about  the  personalities  of  the  customers  and  they  should  have  dealt  with  them.  We  were  surprised,  given  the  low 
fidelity  of  the  interface,  that  both  middle  and  high  school  students  responded  enthusiastically  as  to  the  authenticity. 

Are  the  cognitive  demands  in  the  learning  environment  authentic?  The  next  consideration  in  a discussion  of 
authenticity  is  “Are  the  cognitive  demands  authentic?’  That  is,  does  the  program  demand  the  use  of  a problem 
solving  model  that  is  similar  to  the  one  a real  bank  teller  or  customer  service  representative  would  have  to  use  on  the 
job?  The  Chelsea  Bank  proves  to  be  divided  in  this  area. 

• Learner  conhol  of  decision  making.  The  students  arc  in  control  of  the  decision-making  process  throughout 
the  simulation.  Once  the  customer  provides  the  transaction  the  students  arc  dealing  with,  they  decide  what 
they  need  to  know  and  which  resources  are  appropriate.  This  is  a student-controlled  environment  without 
any  feedback.  The  learner  is  responsible  for  analyzing  the  problem  and  using  the  available  resources  to  find 
a solution.  This  empowerment  of  the  learner  in  the  decision  making  process  is  a real  strength  of  the 
software. 

• Pretraining  and  hints.  The  Teacher’s  Guide  provides  guidance  for  leaching  vocabulary  and  raising  salient 
issues  before  beginning  the  scenario.  In  the  context  of  the  computer  delivery,  there  is  also  a memo  which 
provides  a hint  about  the  salient  issue.  These  kinds  of  hints  and  pretraining  change  the  demands  on  the 
learners  so  that  they  no  longer  have  to  develop  the  strategics  for  working  in  an  authentic  environment, 
rather  they  learn  to  seek  and  interpret  hints  - a different  activity  from  problem  solving  in  the  sense  wc  arc 
trying  to  promote.  For  example,  by  pretraining  vocabulary,  wc  have  removed  the  necessity  for  self- 
monitoring  of  understanding  on  the  part  of  the  students.  They  have  been  spoon-fed  that  which  they  might 
not  have  known  and  can  move  on  to  the  simulation  and  not  think  about  it  anymore. 
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• Multiple  choice  solutions.  The  use  of  multiple  choice  answers  removes  the  need  for  hypothesis  formation 
and  problem  definition  by  the  students.  The  students  can  go  immediately  to  the  solution  options  to  identify 
relevant  variables  and  outcomes  to  consider.  Even  if  the  teacher  prevents  this  shortcut  tot  he  answers,  the 
students  still  know  that  they  do  not  need  to  formulate  their  own  response  to  the  situation.  Rather,  they  need 
only  have  a “sense”  of  what  the  right  thing  to  do  is.  There  is  no  need  for  a rationale.  In  our  work  with  one 
of  the  other  Classroom  Inc.  Simulations,  we  found  that  students  who  described  the  simulation  problems  as 
being  pretty  easy  were,  at  times,  completely  unable  to  provide  a problem  statement  or  reasonable  solution 
to  similar  problems  when  taken  out  of  the  multiple  choice  context. 

We  do  see  some  benefit  from  the  use  of  multiple  choice  answers  as  a scaffolding  device  for  the  students. 
An  effective  problem  solving  process  is  to  generate  hypotheses  on  what  might  be  wrong  and  how  to 
respond  to  a customer  - exactly  the  sort  of  thinking  reflected  in  the  multiple  choice  answers.  However,  the 
students  should  be  asked  to  generate  their  own  hypotheses  in  addition  to  those  in  the  multiple  choice 
answers.  Then  the  students  should  be  asked  to  evaluate  each  of  their  hypotheses  to  choose  the  one  that  is 
the  most  correct  and  ethical.  There  should  be  a gradual  transition  throughout  the  simulation  moving  the 
students  away  from  the  multiple  choice  answers  altogether  so  that  by  the  later  scenarios,  the  students  are 
generating  and  evaluating  only  their  own  hypotheses  and  providing  their  own  solutions  to  the  answers. 

• Lack  of  interaction  with  customers.  One  of  the  stated  objectives  for  this  simulation  has  to  do  with  the 
development  of  interpersonal  skills  through  interaction  with  customers.  The  fact  that  there  is  virtually  no 
interaction  with  the  customers  truly  limits  the  learning  that  can  occur  with  regard  to  this  objective.  While 
the  interpersonal  problems  arc  authentic,  the  cognitive  demands  on  the  students  with  regard  to  interpersonal 
skill  are  not.  Given  the  low  cost  computing  environment  in  this  simulation,  it  is  unclear  to  us  how 
interactions  could  be  incorporated.  This  is  one  of  the  many  instances  where  follow-up  discussions  and  role 
plays  could  help  fill  in  the  shortcoming  of  the  technology. 

Does  scaffolding  help  to  focus  the  learning  objectives?  There  are  a few  issues  worth  exploring  for  this  discussion. 

• Lack  of  time  constraints.  One  of  the  most  effective  uses  of  scaffolding  in  this  program  is  in  removing  the 
time  constraints  that  a bank  employee  would  normally  face  in  making  a decision.  The  goal  is  to  engage  the 
students  in  the  problem  so  they  begin  to  understand  the  types  of  transactions,  strategies  for  determining 
how  to  complete  the  transaction,  and  the  development  of  skills  needed  to  carry  out  that  analysis.  Removing 
the  time  constraints  places  emphasis  on  developing  these  skills. 

• Simplified  banking  tools.  The  resources  are  representative  of  what  the  employees  uses  but  they  arc  greatly 
simplified.  It  is  a question  of  the  skill  levels  of  the  students  as  to  whether  the  materials  arc  so  simple  as  to 
detract  from  the  problem  solving.  For  instance,  the  student  is  presented  with  two  ' ossible  databases  of 
information  to  use  in  working  with  corner.  They  can  do  a credit  check  or  they  can  look  at  account 
balances.  The  program  has  only  included  the  names  of  the  customers  who  need  to  have  their  credit  checked 
in  the  credit  check  database  and  all  of  the  customers  in  the  simulation  who  currently  have  accounts  at  the 
bank  in  the  account  information  screen.  By  doing  this,  the  programmers  have  prevented  students  from 
realizing  the  difference  between  the  two  kinds  of  information.  In  the  students  wc  observed  in  several 
classrooms,  we  continually  saw  students  going  to  the  credit  check  screen  to  verify  account  information. 
When  the  person  they  were  checking  on  did  not  appear  in  the  database,  the  students  cither  went  to  the 
accounts  screen  or  assumed  that  the  customer  did  not  have  an  account  at  the  bank.  Neither  of  these 
situations  is  promoting  the  kind  of  thoughtful  approach  to  resources  the  students  need  to  be  developing. 

• Truncated  task.  The  simulation  takes  the  student  only  to  the  decision  making  process.  This  supports  the 
analysis  of  the  bank  transaction  - but  there  is  no  time  spent  on  the  implementation  of  the  solution. 
However,  developing  interpersonal  skills  requires  an  interchange  between  the  learner  and  the  customer  - 
the  learner  needs  to  be  able  to  try  out  different  ways  of  responding  to  customers  to  sec  what  the  impact  is. 
Thus,  this  task  truncation  supports  the  banking  objectives,  but  not  the  interpersonal  skills  objectives. 

Does  coaching  promote  learning  rather  than  simply  directing  or  correcting  performance?  There  is  no  coaching 

in  The  Chelsea  Bank  and  there  are  no  clear  guidelines  give  to  teachers,  cither  in  the  Teachers  Guide  or  in  the 
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training  they  receive,  on  how  to  coach  the  students  during  their  work  on  the  scenarios.  As  a consequence,  there  is  no 
requirement  to  any  more  than  guess.  We  have  seen  in  our  three  years  of  working  with  this  product  that  students  are 
indeed  engaged  in  their  work  on  the  scenarios.  However,  they  receive  little  or  no  support  to  help  them  evaluate  the 
quality  of  their  work.  Indeed,  there  is  rarely  any  attempt  to  coach  a problem-solving  process  with  regards  to  banking 
practices,  interpersonal  skills,  ethics,  or  working  in  groups  (Hawley  & Duffy,  1997).  Teachers  seem  to  have  no 
sense  for  what  the  problem  solving  process  might  look  like  and  students  do  not  become  involved  with  the  evaluation 
of  their  own  problem-solving  skills  in  the  course  of  their  work  on  the  simulation.  Teachers  often  reported  to  us  that 
they  felt  that  if  the  students  were  engaged  and  working  together,  they  were  learning  about  problem  solving  and 
groupwork. 

We  are  starting  to  be  encouraged  by  a slow  shifting  in  the  teachers  who  are  using  this  program.  In  our 
interviewing  of  teachers  over  the  past  two  years,  we  have  seen  a variety  of  opinions  on  teacher  role  and 
interpretations  of  what  the  roles  mean  as  far  as  the  teacher's  job  is  concerned.  It  is  apparent  that  the  teachers  have 
absorbed  the  jargon  of  what  they  arc  supposed  to  be  doing.  When  as  about  their  role  during  the  simulation,  they 
offered  explanations  such  as: 

You  have  to  help  the  students  think  through  their  thoughts  and  their  decisions  about  what  they're  doing  and 
why  they  are  doing  it.  They  know  what  they  are  doing.  But,  to  help  them  to  understand  why  they  are  doing 
it  is  really  very  important.  So,  by  walking  around  the  room,  even  during  their  pre  or  post-scenario 
discussions,  a lot  of  "Why?"  questions  need  to  come  up.  Because  then  the  teacher  is  better  able  to 
understand  why  the  students  are  doing  the  things  they  are  doing  even  when  they  are  not  doing  The  Chelsea 
Bank.  So,  it  helps  the  teacher  to  better  understand  where  the  kids  are  and  what  they  are  thinking  about. 
(Hawley,  1997) 

However,  the  implementation  of  these  beliefs  is  a slow  one  going  against  the  traditions  of  classroom 
teachers.  It  is  the  facilitation  of  these  skills  that  the  professional  training  being  done  by  Classroom  Inc.  is  starting  to 
really  focus  on. 

Does  reflection  support  abstracting,  synthesizing,  and  extending  the  learning?  Too  often  we  are  involved  in 
projects  where  the  emphasis  is  on  doing,  with  little  encouragement  to  step  back  from  the  doing  and  think  about  what 
was  learned,  what  still  needs  to  be  learned,  and  where  it  might  be  useful.'  In  our  minds  this  is  critical  to  the  learning 
process  for  it  is  the  abstracting  or  indexing  of  what  was  learned  that  makes  the  experience  useful  in  the  future. 

In  The  Chelsea  Bank  wc  found  two  kinds  of  reflective  activities:  a post-decision  writing  activity  that  is  part 
of  the  software  and  certain  teacher-directed  activities  outlined  in  the  Teachers  Guide.  The  computer-based  reflective 
activity  involves  the  student  stating  the  problem  they  just  solved,  naming  the  banking  principle  that  helped  them 
solve  the  problem,  and  talking  about  the  impact  their  decision  will  have  ont  the  bank,  ont  he  customer,  and  on 
themselves  as  the  teller  or  customer  service  representative.  Consistent  with  the  goals  of  reflection,  the  student  may 
leave  the  reflective  activity  and  return  to  the  decision  making  screen  or  review  any  of  the  materials  presented  in  the 
scenario  at  any  point.  The  decision  itself  may  be  changed  at  any  point  up  until  the  reflection  activities  are  completed 
and  the  students  submit  them  as  their  final  answer.  This  ability  to  change  the  decision  at  any  point  during  the 
reflective  activities  is  one  of  the  excellent  features  of  this  simulation. 

The  ability  to  act  on  this  reflection  is  wonderful,  however,  the  reflective  activity  itself  is  limited  for  three 
reasons.  First,  the  same  questions  arc  asked  each  time  which  prevents  them  from  being  contextual iz.cd  to  the 
situation. Second,  there  is  no  established  purpose  for  the  reflection  - to  the  students  it  appears  simply  as  a writing 
task.  Finally,  the  focus  of  the  reflection  is  on  the  outcome  and  not  on  the  process.  There  is  no  attempt  to  abstract  the 
problem-solving  process  or  help  students  identify  the  key  principles  guiding  decision  making. 

The  post-scenario  reflective  activities  outlined  in  the  Teachers  Guide  hold  considerable  potential.  However, 
these  activities  lend  to  focus  more  on  interpersonal  skills  and  also  tend  to  move  toward  new  situations  rather  than 
reflecting  on  the  situation  that  just  occurred.  While  both  of  these  are  valuable  activities  for  other  reasons,  such  as 
transfer  of  skills  and  development  of  social  empathy,  they  do  not  support  the  reflective  activity  that  is  at  the  heart  of 
learning. 

Is  the  environment  engaging?  In  the  case  of  The  Chelsea  Bank , as  well  as  the  other  Classroom  Inc.  simulations, 
this  is  one  of  the  most  striking  things.  Even  though  the  simulations  arc  relatively  low-fidelity  to  allow  for  the  widest 
possible  audience,  the  kids  become  totally  engaged  in  the  simulation.  In  fact,  it  was  this  high  level  of  engagement 
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that  first  struck  our  research  team  - in  every  class  visited,  the  kids  were  huddled  in  groups  actively  working  on 
problems.  Even  in  our  discussion  with  the  students  the  level  of  engagement  was  apparent.  The  students  talked  about 
the  characters  as  if  they  were  real  people  and  the  decisions  as  if  they  were  really  made. 


Summary  and  Conclusions 

Too  often  computer  simulations  arc  viewed  as  standalone  applications  without  recognizing  the  critical  role 
that  teachers  play  in  their  successful  implementation.  This  is  true  even  though  we  have  a long  history  of  data 
indicating  that  the  teacher  variable  can  override  virtually  any  “technology”  variable  in  the  educational  process.  It 
should  also  be  clear  from  this  discussion  that  the  teacher  plays  a critical  role  in  coaching  the  students  during  the 
simulation  and  in  supporting  reflection.  The  simulation  can  provide  a “doing”  environment  but  we  would  argue  that 
the  situated  role  of  the  teacher  is  essential  to  creating  a learning  environment  (Chaney,  1996;  Duffy  & Chaney,  in 
press).  With  the  teacher  support  in  coaching  and  reflection,  simulations  which  are  currently  viewed  as  games 
because  of  the  focus  on  doing,  could  become  effective  educational  tools. 

The  Chelsea  Teacher’s  Guide  offers  no  guidance  on  what  the  teacher  should  be  doing  during  the 
simulation.  In  our  discussions  with  teachers,  there  was  a frequently  expressed  opinion  that  the  simulation  was 
successful  if  the  students  could  work  through  the  scenarios  with  assistance.  There  was  also  a belief  that  simplv  by 
engaging  in  this  problem  solving  environment  and  working  in  groups  the  students  were  learning  transferable 
problem-solving  and  groupwork  skills.  Wc  would  like  to  end  this  discussion  by  emphasizing  that  thinking  about 
coaching  and  reflection  while  students  work  in  small  groups  is  no  easy  task.  This  represents  an  important  area  for 
future  research,  some  of  which  wc  are  currently  involved  in,  which  explores  what  the  role  of  the  teacher  should  be 
and  what  other  scaffolding  activities  might  maximize  the  learning  potential  of  these  simulation  environments. 
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Abstract 

One  form  of  learning  environments  that  continue  to  draw  attention  and  interest  are  those  open-ended  in 
nature.  Open-ended  learning  environments  (OELEs)  are  learner-centered;  the  instructor  plays  the  role  of  a 
facilitator  rather  than  that  of  the  focal  point  for  learning.  While  the  opportunities  afforded  by  OELEs  fur  enhancing 
learning  are  substantial,  considerable  challenges  arise  in  their  creation  and  implementation.  The  purpose  of  this 
paper  is  to  present  a framework  and  model  for  the  design  of  OELEs , We  first  present  an  overview  of  OELEs.  Next, 
assumptions  and  issues  associated  with  the  design  of  OELEs  are  discussed.  Following  the  establishment  of  a 
framework  for  understanding  the  challenges  and  opportunities  afforded  by  OELEs,  a theoretical  model  for 
designing  OELEs  is  introduced.  Finally,  guidelines  will  be  offered  to  assist  in  the  use  of  the  model  for  the  design 
and  development  of  learning  environments  based  on  open-ended  constructs. 

One  form  of  learning  environments  that  continue  to  draw  attention  and  interest  arc  those  open-ended  in 
nature.  Open-ended  learning  environments  (OELEs)  afford  and  support  exploration  and  experimentation;  problem 
solving,  critical  thinking,  and  multiple  perspectives  are  essential  processes  in  these  environments  [for  a more 
complete  description  of  OELEs,  see  Hannafin,  Hall,  Land  & Hill,  1994].  OELEs  arc  learner-centered;  the  instructor 
plays  the  role  of  a facilitator  rather  than  that  of  the  focal  point  for  learning.  Individual  experiences  and  contexts  are 
emphasized  as  learners  cultivate  cognitive  processes  to  support  understanding  in  OELEs.  The  development  of 
strategies  and  processes  are  focal  points  for  demonstrating  growth  in  understanding  (Land  & Hill,  1997;  Papert, 
1993). 

While  the  opportunities  afforded  by  OELEs  for  enhancing  learning  are  substantial,  considerable  challenges 
arise  in  their  creation  and  implementation.  OELEs  come  in  many  varieties  (simplistic  and  focused;  complicated  and 
limitless)  and  can  be  manifested  in  several  settings  (face-to-face  instruction,  distance  learning  environments, 
computer  assisted  instruction,  etc.).  Like  most  learning  situations,  well-designed  and  developed  OELEs  have  the 
ability  to  empower,  liberate,  and  expand  the  orientations  of  their  participants  [for  related  discussions  on  the  value  of 
sound  instructional  design  in  other  learning  situations,  sec  Moore  & Kcarsley  (1996);  Willis  (1993)].  At  the  same 
time,  OELEs  are  demanding,  and  can  be  disorienting,  even  unsettling,  for  those  engaged  in  these  environments.  The 
learners,  as  well  as  the  instructor,  arc  placed  in  roles  not  traditionally  held  in  formal  learning  environments; 
furthermore,  the  processes  associated  with  these  environments  (problem  solving,  critical  thinking,  etc.)  arc  ones 
which  demand  considerable  work  by  all  participants  (Hannafin,  Hill,  & Land,  1997).  Designing  and  developing 
learning  environments  that  empower  the  user,  are  intuitive  and  self-evident,  and  inclusive  in  orientation  is  a 
formidable  challenge  (Norman,  1988). 

The  purpose  of  this  paper  is  to  present  a framework  and  model  for  the  design  of  OELEs.  We  first  present 
an  overview  of  OELEs.  Next,  assumptions  and  issues  associated  with  the  design  of  OELEs  arc  discussed. 
Following  the  establishment  of  a framework  for  understanding  the  challenges  and  opportunities  afforded  by  OELEs, 
a theorcti  .al  model  for  designing  OELEs  is  introduced.  Finally,  guidelines  will  be  offered  to  assist  in  the  use  of  the 
model  for  the  design  and  development  of  learning  environments  based  on  open-ended  constructs. 
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Overview  of  QELEs 

OELEs  coordinate  application  of  a variety  of  tools  and  resources  for  use  in  addressing  situated,  authentic 
problems  (Hannafin,  Hill,  & Land,  1997).  Technology  is  often  used  as  a mediator  of  the  process,  providing  learners 
with  electronic  means  to  search  vast  databases  of  resources  and  manipulate  variables  and  concepts.  Yet, 
fundamentally,  OELEs  support  student-centered  understanding  — the  learner  is  at  the  center  of  the  environments 
both  in  terms  of  decisions  for  using  the  available  resources  and  as  the  party  with  the  primary  responsibility  for 
learning.  As  such,  OELEs  have  relied  heavily  upon  theoretical  views  from  constructivists,  who  assert  that 
understanding  is  best  achieved  when  it  is  individually  driven  by,  or  constructed  by,  the  learner  (Jonassen,  1991; 
Phillips,  1995).  In  essence,  OELEs  support  learner-centered  construction  with  opportunities  to  relate  new 
knowledge  to  personal  experiences  from  a problem-based,  activity-intensive  orientation.  Technologically-based 
tools  and  resources  are  used  as  means  to  support  the  constructive  process. 

OELEs  provide  interactive,  complimentary  activities  that  revolve  around  problem-based  contexts  and 
support  individual  sense-making  processes  (Hannafin,  1992;  Hannafin,  et  aL,  1997).  OELEs  are  comprised  of 
several  key  characteristics  that  enable  them  to  support  divergent  student  needs  and  establish  the  conditions  for 
enriching  thinking.  Several  of  these  key  characteristics  include  the  following:  (1)  use  of  meaningful,  complex 
contexts  (2)  provision  of  tools  and  resources;  (3)  learner  reflection  and  self-monitoring;  and  (4)  social,  material,  or 
technological  scaffolding. 

Use  of  Meaningful,  Complex  Contexts 

One  common  characteristic  of  OELEs  is  the  use  of  broad,  problem-based  contexts  that  form  the  adhesive 
for  all  conceptual  activity.  Problem-based  contexts  serve  three  primary  purposes:  (1)  to  orient  learners  to  the 
activity;  (2)  to  provide  an  guide  for  applying  what  is  known  and  evaluating  what  is  not  known;  and  (3)  to  assist 
learners  in  connecting  formal  concepts  to  everyday  applications  of  them. 

Scenarios  or  cases,  for  instance,  are  often  used  to  guide  learners  in  exploring  the  complexities  of  a topic. 
Such  contexts  often  focus  on  everyday  problems  (e.g.,  environmental  pollution  for  studying  chemistry  concepts; 
swimming  pools  for  studying  buoyancy  and  water  displacement;  corporate  cases  to  study  management  principles; 
real  patient  cases  for  learning  about  radiology,  etc.).  Everyday  problems  are  used  to  increase  the  likelihood  that 
learners  will  readily  identify  how  concepts  can  be  applied  in  a given  setting.  Complex  contexts  are  used  as  a way  to 
assist  learners  in  thinking  about  the  content  in  ways  that  are  consonant  with  a community  of  practice.  It  is  not  likely, 
for  instance,  that  scientists  learn  about  science  by  reading  and  memorizing  laws;  rather,  they  use  an  inquiry-based 
approaches  for  hypothesizing  and  predicting  outcomes  based  upon  a collection  of  empirical  data.  OELEs  strive  to 
mirror  the  holistic  thinking  practices  of  experts,  within  boundaries  accessible  to  novice  learners. 

Complex  contexts  provide  an  anchor  fc.-  making  sense  out  of  discrete  pieces  of  information.  Rather  than 
memorize  or  learn  content  in  absence  from  an  applied  context,  information  is  learned  as  a result  of  needing  to  know 
it  in  order  to  solve  the  problem.  Problem-based  approaches  in  medical  schools,  for  instance,  teach  learners  to  use 
hypothesis-driven  approaches  — i.e.,  generate  hypotheses  about  a case  to  explain  the  data  with  incomplete 
understanding  of  the  related  knowledge  base  (Hmelo,  Gotterer,  & Bransford,  1997).  Once  hypotheses  are  generated, 
data  is  searched  and  selected  that  confirm,  formalize,  and/or  refute  the  hypothesis.  Consequently,  new  data  or 
information  becomes  meaningful  as  its  potential  for  use  is  evaluated  in  light  of  a driving  context  and  hypothesis. 
OELEs  use  complex,  meaningful  contexts  to  assist  learners  in  building  formative  theories  and  to  then  search  for  new 
information  to  confirm,  elaborate,  or  refute  the  hypothesis. 

Provision  of  Tools  and  Resources 

OELEs  use  tools  and  resources  to  assist  learners  in  accessing  both  sources  and  perspectives  related  to  the 
content  under  study.  Often,  a range  of  resources  arc  provided  that  serve  as  repositories  of  information  (c.g.,  CD- 
ROMs,  encyclopedias)  that  can  be  brought  to  bear  to  solve  a particular  problem.  For  instance,  problem-based 
approaches  for  medical  school  training  may  incorporate  databases  of  resources  such  as  patient  X-Rays,  disease 
diagnosis  references,  or  patient  histories.  Students  using  a microworld  on  physics  concepts  may  access  resources 
about  formal  concepts  such  Newton’s  Laws,  or  a database  of  student  perspectives  regarding  how  the  concepts 
manifest  in  everyday  life  (e.g.,  hitting  a baseball;  riding  a bicyclr,  slowing  down  in  a car,  etc,). 

Environments  can  also  be  designed  to  facilitate  the  construction  of  resources  by  learners.  For  instance, 
students  can  learn  about  fractions  by  designing  and  constructing  educational  software  for  leaching  younger  children 
about  fractions  (Hard  & Papert,  1991 ).  Similarly,  tools  such  as  Intermedia  utilize  a networked  multimedia  system 
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where  learners  construct  a "web”  of  concepts,  and  share  them  communally  with  other  students.  The  use  of  resources, 
or  opportunities  to  construct  resources,  provides  a rich  environment  forextending  understanding. 

Tools  for  constructing  and  manipulating  understanding  arc  used  to  promote  learning  that  is  more  concrete 
and  capable  of  being  tested.  Tools,  such  as  spreadsheets  or  word  processors,  provide  opportunities  for  user-centered 
activity.  In  learning  environments,  tools  help  learners  to  manipulate  features  and  processes.  Some  tools,  such  as 
those  found  in  simulations  and  microworlds,  aliow  learners  to  manipulate  concepts  by  varying  parameters  and/or 
physical  models  (e.g.,  vary  force  and  direction  of  an  object  in  space,  [Richer,  1992]).  Computerized  tools  can  be 
used  to  select  text  for  electronic  notebooks,  create  hyperlinks  between  sources  of  information,  or  perform 
calculations  (Hannafin,  1992).  Tools  allow  learners  to  test  complex  theoretical  concepts  in  concrete  ways  (Hannafin 
etal.,  1994). 

Learner  Reflection  and  Self-Monitoring 

The  student-centered  learning  process  hinges  upon  the  learner’s  ability  to  monitor  learning  needs  and  to 
place  into  action  planning  and  evaluation  approaches.  OELEs  typically  emphasize  activities  that  induce  and 
facilitate  reflection  on  the  learning  process.  The  CSILE  environment,  for  instance,  is  designed  to  facilitate 
metacognitive  thinking  through  the  use  of  prompts  to  generate  questions,  hypotheses,  or  theories  (Scardamalia  ct  al., 
1989).  Other  environments  facilitate  reflection  of  scientific  inquiry  or  critical  thinking  skills  by  embedding  activities 
that  induce  connection  of  hypotheses  to  observations  (Lewis,  Stern,  & Linn,  1993).  By  virtue  of  their  design, 
OELEs  require  complex  thinking  skills  and  a variety  of  strategic  and  evaluative  processes.  For  this  reason,  OELEs 
guide  learners  in  using  these  techniques  by  embedding  the  reflective  requirements  of  the  activity  into  the 
environment  itself. 

OELEs  often  require  the  creation  of  end-products  that  make  learner  reasoning  overt.  It  is  not  enough,  for 
instance,  for  a learner  to  simply  manipulate  variables  in  a microworld  or  develop  hypotheses  without  understanding 
why  they  are  relevant.  Instead,  OELEs  require  learners  to  communicate  what  they  have  learned  through  the 
development  of  “artifacts”  that  reflect  both  the  product  of  their  understanding  and/or  underlying  argumentation. 
Project-based  approaches,  for  instance,  often  revolve  around  the  creation  of  learner-generated  multimedia  products. 
Thus,  a preservice  teacher  learning  how  to  incorporate  technology  into  the  curriculum,  might  develop  an  example 
lesson  indicating  how  they  would  use  technology  in  the  classroom.  Accompanying  this  product  might  also  be 
documentation  regarding  how  it  solves  an  identified  problem  and  how  technology  is  an  instrumental  part  of  the 
solution.  Consequently,  reflective  activity  is  supported  through  both  en-route  and  end-product  requirements  of  the 
activity.  En-route  reflection  is  necessary  for  learners  to  ask  driving  questions,  identify  “nceds-to-know,”  and 
implement  strategic  \ <ans  during  the  open-ended  learning  process.  End-product  reflection  is  necessary  to  justify  or 
argue  what  has  been  constructed. 

Social,  Material,  or  Technological  Scaffolding 

OELEs  rely  on  the  learner  to  direct  the  learning  process,  formulate  goals,  and  interpret  events  within  the 
environment.  Social,  material,  or  technological  support  is  also  provided  to  assist  learners  in  the  knowledge 
construction  process.  Many  OELEs,  for  instance,  utilize  teacher-student  and  student-student  interactions  to  model 
or  scaffold  reflection  and  performance  (Hannafin  & Land,  1997;  Palincsar  & Brown,  1984;  Scardamalia  & Bereitcr, 
1985).  In  such  environments,  teachers  and  students  coach,  model,  and  share  strategics  within  a problem  context 
(Collins,  Brown,  & Newman,  1989).  Learners  contribute  to  continually  evolving  archives  of  knowledge  that  arc 
shared  and  used  as  the  basis  for  evaluating  individual  understanding  (Scardamalia  et  al.,  1989).  Such  scaffolding 
emphasizes  the  sharing  of  sense-making  processes  and  the  progressive  negotiation  of  meaning. 

Technology  is  also  used  as  a way  of  scaffolding  performance.  Technology-based  environments  often 
“...provide  models,  opportunity  for  higher  level  thinking,  and  metacognitive  guidance...  in  a learner’s  zone  of 
proximal  development”  (Salomon,  Globcrson,  & Guterman,  1989,  p.  620).  That  is,  technology  is  used  in  ways  to 
support  understanding  that  would  be  difficult,  if  not  impolitic,  to  support  otherwise.  For  instance,  visualization 
tools  used  in  microworlds  such  as  Geometer's  Sketchpad  anu  Interactive  Physics  allow  learners  to  construct  models 
or  objects  and  rotate  and  manipulate  them  in  order  to  test  their  parameters.  Technological  tools  can  scaffold 
opportunities  for  learning  by  altering  both  the  experiences  available  to  learners  and  the  cognitive  ! equipments  of  the 
learning  task  (Salomon,  1986). 
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Assumptions  and  Issues  Associated  with  the  Design  of  OELEs 

A variety  of  isolated  case  studies  of  OELEs  have  been  detailed  that  discuss  the  design  and  development 
considerations  specific  to  the  given  environment  (see  for  example,  Cognition  and  Technology  Group  at  Vanderbilt, 
1992;  Richer,  1992;  Lewis  et  al.,  1993).  Research  on  these  environments  has  been  promising,  but  often  detail  the 
seemingly  unique  requirements  and  features  of  a particular  environment.  General  design  guidelines  and  heuristics 
have  occasionally  been  offered,  but  little  has  been  done  to  synthesize  the  common  features  and  assumptions 
underlying  these  environments,  nor  to  present  a unified  theory  of  design.  Consequently,  apart  from  isolated 
conceptions  of  OELEs,  little  is  understood  regarding  how  tc  design  learner-centered  environments  that  cross  content 
areas  and  specific  uses  of  technology  (Hannafin  & Land,  1997). 

The  concept  underlying  OELEs,  however,  is  not  limited  to  solely  one  kind  of  technology-based 
environments.  Rather,  OELEs  comprise  many  forms,  often  with  few  discernible  similarities.  This  makes  it  difficult 
to  identify  common  characteristics  to  use  as  the  basis  for  design.  This  section,  then,  will  discuss  common 
assumptions  that  are  manifested  across  diverse  environments. 

Assumption  One:  Understanding  is  best  achieved  when  situated  in  relevant  contexts. 

One  assumption  underlying  use  of  OELEs  is  that  knowledge,  context,  and  process  arc  inextricably  lied 
(Brown,  Collins,  & Duguid,  1988).  This  implies  that  learning  of  discrete  information,  decontextualized  from  its 
application,  will  remain  “inert”  (Whitehead,  1929)  or  unable  to  be  used.  Rather  than  isolating  information,  OELEs 
embed  relevant  knowledge  within  problem  contexts.  Thus,  learning  of  specific  information  takes  place  as  a 
consequence  of  needing  to  know  it  and  apply  it  to  solve  the  current  problem.  Science  Vision  (Tobin  & Dawson, 
1992),  for  instance,  immerses  students  in  learning  science  concepts  through  use  of  an  authentic  context.  Students 
learn  about  force  and  motion  concepts,  for  instance,  by  using  them  to  build  a virtual  roller  coaster.  They  learn 
chemistry  concepts  by  resolving  problems  related  to  a polluted  river.  Information  that  is  anchored  in  relevant  and 
situated  contexts  is  viewed  as  integral  to  learning  how  and  why  information  is  meaningful. 

Furthermore,  situating  knowledge  within  realistic  practices  requires  learners  to  model  the  processes  that 
they  may  ultimately  practice.  That  is,  the  process  required  to  apply  information  to  solve  problems  is  consonant  with 
the  manner  in  which  experts  or  practitioners  use  them.  In  this  sense,  the  process  of  learning  is  not  disconnected 
from  the  process  by  which  knowledge  and  skills  will  be  ultimately  used.  Preservice  teachers,  for  instance,  typically 
lake  courses  on  learning  how  to  integrate  technology  into  the  curriculum.  An  authentic  context  for  learning  these 
types  of  skills  would  incorporate  learning  of  the  technology  with  learning  how  to  apply  it  in  the  classroom.  Yet,  it  is 
more  typical  that  they  learn  the  discrete  “how  to’s”  of  using  technology,  without  learning  how  to  think  about 
conceptualizing  technology-based  learning  experiences.  Using  situated  contexts  helps  to  assure  that  knowledge  is 
not  separated  from  either  the  process  or  context  of  applying  it. 

Assumption  Two:  Learners  must  take  more  responsibility  for  monitoring,  and  reflecting  upon,  the  learning 
process. 

OELEs  assume  that  learners  have  the  metacognitivc  awareness  to  take  direction  of  the  planning, 
implementing,  and  evaluation  process.  Through  guidance  from  the  system  learners  are  provided  with  hints, 
orienting  scenarios,  help  options,  and  direct  instruction  if  needed.  Presumably,  with  more  opportunities  to  make 
choices  regarding  what,  when,  and  how  to  learn,  learners  evolve  greater  responsibility  for  their  learning.  Evidence 
also  suggests  that  learners  arc  becoming  increasingly  compliant  in  their  thinking  - seeing  the  learning  task  as  one  of 
matching  their  ideas  a :d  theories  of  an  external  agent  (typically  a teacher)  (McCaslin  & Good,  1992).  For  these 
reasons,  OELEs  provide  learners  with  opportunities  to  derive  and  test  individual  sense-making  processes. 

To  be  effective  during  open-ended  learning,  learners  must  monitor  their  thoughts  and  actions.  Learners 
interact  based  upon  mctacognitive  awareness  of  their  understanding  and  the  perceived  need  to  validate  or  challenge 
their  understanding  (Perkins,  1993).  This  includes  decisions  to  pursue  additional  practice,  search  for  definitions  or 
information,  test  a hypothesis,  create  a “what  if’  scenario,  or  take  notes.  Learners  must  be  able  to  locate,  select, 
organize,  integrate,  and  use  relevant  information  if  they  arc  to  generate  products  and/or  understanding.  Similarly, 
learners  must  evaluate  ihc  adequacy  of  their  approaches  during  open-ended  learning  (Belmont,  1989).  Learners  are 
assumed  capable  of  meeting  these  mctacognitive  requirements  if  they  arc  supported  by  system- learner,  leaclicr- 
lcarner  and  learner-learner  interactions. 
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Assumption  Three:  Understanding  is  best  supported  when  learners  connect  personal  experiences  with 
formal  concepts. 

One  assumption  of  OELEs  is  that  personal  beliefs,  experiences,  and  conceptual  schemata  support  current, 
as  well  as  provide  the  foundation  for  new,  understanding  (Hannafin,  1992).  Background  knowledge  and  experience 
form  the  conceptual  referent  within  which  new  encounters  are  organized  and  assimilated  (Piaget,  1976). 
Background  context  influences  the  choices  learners  make  in  the  environment,  the  extent  to  which  they  persevere, 
and  the  types  of  goals  they  set.  Accordingly,  learner  use  of  prior  experiences  as  a referent  for  understanding  is 
foundational  to  the  design  and  development  of  OELEs. 

OELEs  often  utilize  problem  contexts  that  are  accessible  to  learners’  everyday  experiences.  Rather  than 
present  information  in  the  abstract,  it  is  presented  within  rich,  concrete  experiences  that  can  be  manipulated, 
compared,  and  explored.  By  allowing  learners  to  have  concrete  experiences  with  phenomena,  they  are  given  the 
opportunity  to  reshape  ongoing  theories  that  are  based  upon  personal  experiences.  For  instance,  the  ErgoMotion 
environment  supports  learners  in  the  development  a virtual  roller  coaster,  in  order  to  explore  basic  mechanics 
principles.  It  is  assumed  that  learners  will  utilize  their  experiences  in  riding  roller  coasters  as  the  basis  for  making 
decisions  and  making  sense  out  of  the  events  in  the  environment.  As  learners  connect  system  experiences  with  prior 
experiences,  they  access  existing  frameworks  to  be  build  upon.  This  knowledge  can  then  be  used  to  enrich  or 
elaborate  the  formal  concepts  under  study,  as  they  are  integrated  into  prior  experiences.  It  is  commonly  believed 
that  integrating  new  knowledge  with  existing  conceptions  results  in  more  meaningful  learning  (Mayer,  1984). 

Assumption  Four;  Learning  is  a byproduct  of  progressive  negotiation  and  interpretation  of  meanings. 

OELEs  support  personal  theory-building  through  opportunities  to  generate,  test,  and  revise  ongoing 
theories-in-action  (Land  & Hannafin,  1996).  It  is  assumed  that  learners  hold  initial,  although  often  incomplete  or 
naive  beliefs,  that  are  progressively  refined  through  interaction  with  the  environment.  These  beliefs  form  the 
foundation  for  understanding  and  establish  assumptions  that  can  be  tested  while  engaging  the  system,  teacher,  or 
other  students.  Microworlds  and  simulations,  for  instance,  allow  learners  to  generate  and  test  “working  models"  of 
their  understanding.  By  varying  parameters  and  hypothesizing  their  outcomes,  learners  test  assumptions  and  revise 
thinking  based  on  observations. 

OELEs  arc  designed  to  assist  the  leaner  in  the  process  of  angling,  or  considering  multiple  perspectives, 
during  the  learning  process.  Multiple  perspectives  from  teachers,  experts,  or  peers  arc  coordinated  to  form  a 
knowledge  base  from  which  learners  evaluate  multiple  sources  of  meaning.  Bubble  Dialogue  (Language 
Development  and  Hypermedia  Research  Group,  1992),  for  example,  allows  learners  to  share  dialog,  viewpoints,  and 
perspectives.  Similarly,  the  Knowledge  Integration  Environment  (Linn,  1995)  uses  web  technology  to  support 
learners  in  sharing  databases  of  learner-constructcd  evidence  for  evaluating  scientific,  phenomena.  Varied  methods 
and  perspectives  arc  viewed  as  critical  to  developing  deeper,  divergent,  and  more  flexible  thinking  processes. 

Summary 

OELEs  draw  upon  a variety  of  theoretical  foundations  and  methods  in  their  design.  Yet,  despite  the 
differences  among  isolated  approaches,  most  OELEs  share  a common  set  of  key  characteristics  and  assumptions. 
The  common  characteristics  and  assumptions  can  be  used  to  leverage  the  effective  design  of  these  environments. 
What  remains  to  be  discovered  are  “best  practices”  and/or  “best  models”  for  doing  so. 

Theoretical  Model  for  the  Design  of  OELEs 

Key  characteristics  and  assumptions  of  OELEs,  as  well  as  our  experiences  in  the  development  of  OELEs, 
were  used  to  guide  the  creation  of  a theoretical  model  for  the  design  of  OELEs.  The  model  depicted  in  Figure  1 is 
an  example  of  how  elements  within  the  model  overlap  and  build  upon  one  another.  What  follows  is  an  explanation 
of  each  phase  of  the  model.  The  reader  is  reminded  that  this  is  a theoretical  model;  continued  exploration  and  use  of 
the  model  for  designing  and  developing  OELEs  is  needed. 
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Figure  J.  Theoretical  model  for  the  design  ofOELEs. 


Analysis 

The  analysis  phase  leads  (he  way  in  the  design  of  OELEs,  as  it  does  in  most  instructional  design  models 
(Dick  & Carey,  1997;  Kemp,  Morrison  & Ross,  1998;  Smith  & Ragan,  1993;  West,  Farmer  & Wolff,  1991). 
Analysis  in  OELE  design  places  the  designer  in  the  role  of  sleuth,  seeking  to  discover  information  about  the 
environment  in  which  the  OELE  will  be  implemented  (face-to-face  or  other)  and  the  participants  in  the  OELE, 
including  the  learners  and  the  instructor.  OELE  analysis  also  involves  the  exploration  of  the  task,  both  from  the 
learner’s  and  educator’s  perspectives. 

Environmental  analysis.  Environmental  analysis  is  a sub-process  in  the  analysis  phase.  As  defined  by 
Tessmcr  (1991),  environmental  analysis  involves  “...the  analysis  of  the  context  in  which  the  instructional  product 
will  be  employed,  of  the  physical  and  use  factors  of  the  instructional  environment,  and  its  support  environment  (p. 
9).  The  goal  in  environmental  analysis  is  to  describe  where  the  product  will  be  used,  how  it  will  be  used,  and  how  it 
will  be  sustained  in  its  use.  Several  areas  are  explored  during  environmental  analysis. 

1.  The  driving  factors  behind  the  creation  of  the  OELE.  For  example,  one  question  asked  during  the 
environmental  analysis  is  “why  is  the  instruction  important?  In  asking  this  question,  the  designer  seeks  to 
provide  a societal  context  for  the  OELE  (West,  Farmer  & Wolff,  1991).  Examples  of  why  the  instruction 
may  be  important  include:  economic  forces  (c.g.,  need  for  more  knowledge  workers;  need  for  learners  with 
a higher  degree  of  cognitive  strategies),  social  (e.g.,  need  for  stronger  interpersonal  skills),  and  physical 
(c.g.,  need  for  greater  eye-hand  coordination  skills). 

2.  The  viewpoint  taken  in  the  creation  of  the  OELE.  West,  Farmer  & Wolff  (1991)  discuss  three 
viewpoints:  functionalist,  conflict,  and  critical.  Of  the  three,  a critical  viewpoint  is  more  aligned  with  the 
characteristics  and  assumptions  of  OELEs.  A critical  viewpoint  involves  an  emphasis  on  “...the  need  for 
individuals  to  be  critically  aware  of  the  choices  they  make  and  their  implications”  (p.  224)  [see  West. 
Farmer  & Wolff  (1991)  for  a more  thorough  discussion  of  the  viewpoints].  Creating  an  environment  where 
learners  drive  what  occurs  and  also  take  on  an  acceptance  of  this  responsibility,  it  a key  characteristic  in 
OELE  design.  It  is  important  to  note  that  the  educator,  or  learner,  may  come  to  the  OELE  with  another 
viewpoint  toward  learning  (c.g.,  functionalist  or  conflict).  To  become  fully  immersed  in  an  OELE,  it  is 
important  to  evaluate  viewpoint,  and  work  toward  a more  critical  orientation. 

3.  The  structure  of  the  learning  environment.  The  structure  of  the  OELE  determines  the  degree  of 
definition  (well-defined  or  ill-defined)  given  to  the  problems  prcscntcd/crcaicd  in  the  OELE.  Spread  across 
a continuum,  the  problems  in  an  OELE  typically  range  from  moderately  defined  to  ill-defined  [see 
Jonassen  (1997)  for  a thorough  discussion  of  problem  states].  Determining  how  much  structure  will  be  a 
part  of  the  OELE  will  establish  a foundation  for  the  continued  development  of  the  OELE. 

4.  The  purpose  in  creating  the  OELE.  In  creating  a purpose,  the  designer  of  the  OELE  defines  several 
fundamental  elements  in  the  design  process.  These  include: 

• the  problem  being  solved  by  the  OELE  (e.g.,  lack  of  transfer  to  novel  situations;  little 
opportunities  for  problem-solving;  lack  of  argumentation  from  multiple  perspectives,  etc.). 

• the  goal  (both  in  terms  of  content  and  processes)  of  the  OELE  (c.g.,  argue  a position  using  formal 
and  informal  sources  of  evidence;  engage  the  scientific  inquiry  process,  etc.) 
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• the  type  of  learning  the  goal  exhibits  (ala,  Gagne’s  (1972)  type  of  learning  outcome);  and 

• the  function  of  the  OELE  (i.e.,  why  do  the  learners  need  to  know  what  is  being  learned). 

Participants:  Their  Rote  and  Characteristics.  The  next  sub-phase  in  analysis  involves  getting  to  know  the 

participants  who  will  be  taking  part  in  the  OELE.  This  phase  in  the  design  process  also  involves  role  and 
characteristic  identification  for  the  various  participants  (e.g.,  educators,  learners  and  external  experts)  (West, 
Farmer,  & Wolff,  1991 ).  Participant  analysis  can  be  divided  into  several  steps. 

1 . Identification  of  who  will  he  participating  in  the  OELE.  Depending  on  the  development  of  the  OELE, 
the  learner  alone  may  be  engaged  in  the  learning  environment.  Independent  study  courses  provide  one 
example  of  how  an  OELE  may  be  created  with  only  learner  participation. 

Both  the  learner  and  the  educator  may  be  engaged  in  the  OELE.  While  this  may  be  considered  a 
more  “traditional”  orientation  to  how  learning  occurs,  the  roles  that  the  learner  and  educator  play  in  an 
OELE  are  different  than  those  traditionally  played  in  learning  context.  Defining  the  roles  played  is  a 
critical  step  in  OELE  design  and  is  discussed  in  the  next  section. 

A final  consideration  in  terms  of  OELE  participant  identification  relates  to  external  experts.  In 
making  the  decision  related  to  the  inclusion  of  external  experts,  the  OELE  designer  seeks  to  answer  a two- 
part  question:  will  external  experts  play  a part  in  the  OELE,  and,  if  so,  in  what  capacity  and  how  much? 

2.  What  role  will  be  played  by  the  participants  in  the  OELE.  Making  decisions  regarding  who  will  take 
part  in  the  OELE  are  important.  However,  defining  their  roles  hits  at  the  crux  of  a primary  assumption  of 
OELEs:  the  learner  taking  on  a greater  sense  of  responsibility  for  their  learning. 

The  roles  played  by  those  participating  in  the  OELE  are  manifested  in  various  ways.  It  should  be 
noted  that  while  it  is  important  to  define  roles  in  the  analysis  phase,  participant  roles  can,  and  often  do, 
change  during  implementation  of  the  OELE.  One  element  that  may  drive  the  need  for  redefinition  of  roles 
is  the  comfort  level  of  both  the  learner  and  educator  in  working  in  an  OELE.  The  experience  level  of  the 
learner  and  educator  in  engaging  in  OELEs  may  also  affect  role  definition. 

Three  potential  scenarios  can  be  drawn  related  to  role  definition. 

• Scenario  1:  The  learner  has  almost  all  of  the  control  in  the  OELE.  This  control  may  include  (but 
is  not  limited  to)  setting  goals,  selecting  resources,  and  defining  problems  to  be  solved.  In  this 
scenario,  the  learner  is  very  much  engaged  in  self-directed  learning. 

• Scenario  2:  The  learner  and  the  educator  play  somewhat  equal  roles  in  the  OELE.  In  this 
instance,  the  educator  may  have  some  goals/objcc lives  they  consider  essential  for  the  learner  to 
achieve  as  they  engage  in  the  OELE.  The  educator  may  also  have  pre-selected  some  (perhaps  all) 
of  the  resources  to  be  used  in  the  OELE.  However,  the  learner  may  have  a voice  in  determining 
how  or  when  these  goals/objectives  are  achieved,  as  well  as  some  of  the  resources  they  select  for 
achieving  those  goals/objectivcs. 

• Scenario  3:  The  learner  takes  more  of  a leading  role , while  the  educator  plays  a support  role  in 
the  OELE  (i.e.,  socially  scaffolded  learning  environment).  In  this  instance,  the  educator  acts  as  a 
foundation  builder  for  the  learner’s  engagement  in  the  OELE.  The  educator  works  in  setting  the 
stage,  bringing  in  the  various  props  (i.e.,  resources),  and  creating  an  overall  atmosphere  for  the 
OELE.  The  learner  then  acts  as  the  driving  force  in  what,  how  and  when  things  occur  within  the 
OELE.  In  a face-toTace  environment,  this  may  mean  that  the  educator  establishes  a structure  via  a 
syllabus,  recommended  readings,  suggested  projects,  and  scheduled  external  expert  seminars.  The 
learner  then  makes  decisions  regarding  what  problems  to  solve,  and  how  and  when  to  solve  them. 

3.  Relevant  characteristics  of  participants.  The  final  clement  in  participant  analysis  involves  the 
discovery  of  participant  characteristics  which  may  impact  their  involvement  in  the  OELE.  Particular 
areas  to  attend: 

• the  learners  prior  experiences  both  in  terms  of  the  concepts  being  learned  and  the  process 
being  engaged  in  the  OELE; 

• learner  misconceptions  that  may  underlie  learning  of  the  concepts;  and 

• the  educator’s  prior  experience  with  the  kinds  of  conceptual  and  procedural  shortcomings 
learners  experience  in  OELEs. 


Design 

Design  begins  after  analysis  has  been  initiated,  but  docs  not  wait  until  it  is  “finished/’  In  OELEs,  an 
orientation  toward  rapid  design  is  adopted.  This  is  similar  to  the  rapid  prototyping  model  proposed  by  Tripp  and 
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Bichelmeycr  (1990).  According  to  Tripp  & Bichclmeycr,  rapid  prototyping  involves  the  overlapping  of  various 
steps  or  phases  in  the  instructional  design  process  [sec  Figure  1 for  an  example  of  how  this  is  manifested  in  OEL1Z 
design].  This  aligns  the  designer  with  the  learners  and  educators  who  will  be  engaged  in  the  OELE:  in  a problem- 
solving,  critical  thinking  orientation. 

The  design  phase  in  the  creation  of  an  OELE  engages  the  designer  in  creating  a framework  for  the  learning 
environment.  As  in  most  other  design  models  (Dick  & Carey,  1997;  Kemp,  Morrison  & Ross,  1998;  Smith  & 
Ragan,  1993;  West,  Farmer  & Wolff,  1991),  the  OELE  designer  explores  the  creation  of  objectives  or  goals,  helps 
define  instructional  content,  and  engages  in  strategy  specification. 

Defining  objectives.  The  participant  roles  defined  in  the  analysis  phases  will  have  dircci  implications  for 
how  {if)  objectives  arc  defined  by  the  designer  or  by  the  lea/ner.  In  certain  implementations  of  OELEs,  the  designer 
may  establish  overarching  objectives  for  the  environment,  upon  which  the  learner  elaborates,  adapting  them  to  their 
particular  context  and/or  needs.  This  recognizes  the  fact  that  not  all  objectives  or  goals  can  be  determined  in  OELEs 
a priori.  Yet,  the  designer  is  typically  responding  to  some  needs  or  problems  identified  in  the  analysis  phase.  These 
goals  and  objectives  may  be  broader  than  or  different  in  scope  from  traditional  behavioral  objectives.  But,  typically, 
the  designer  is  considering  a collection  of  overarching  goals  regarding  skills,  processes,  or  strategics  that  are  desired. 
In  other  implementations,  the  learner  derives  their  own  objectives  based  on  what  they  hope  to  achieve.  In  either 
instance,  the  defining  of  the  objectives  usually  occurs  within  the  design  phase  — cither  at  the  beginning  (if 
established  by  the  designer)  or  toward  the  end  (if  established  by  the  learner  when  the  OELE  is  implemented)  of  the 
process. 

Instructional  domains.  The  instructional  domains  component  of  the  design  process  refers  to  determining 
the  type  of  learning  outcomc(s)  the  learner  will  engage  in  the  OELE  (Gagne,  1972).  Instructional  designers  have 
used  a variety  of  taxonomies  to  determine  the  type  of  learning  upon  which  the  gcil/objectivc  focuses.  These 
include:  Bloom’s  taxonomy  of  cognitive  outcomes  (Bloom,  Engclhart,  Furst,  Hill,  & Krathwohl,  1956),  a taxonomy 
of  affective  outcomes  (Krathwohl,  Bloom,  & Masia,  1964),  Simpson’s  plan  for  a taxonomy  of  psychomotor 
outcomes  (Simpson,  1966-67)  and  Gagne's  (1972)  taxonomy  of  learning  outcomes  [note:  sec  Driscoll  (1994)  for  a 
more  comprehensive  discussion  of  each  taxonomy].  Gagne’s  (1972)  taxonomy  is  the  most  comprehensive,  and 
establishes  five  categories  of  learning  outcomes:  verbal  information,  intellectual  skills,  cognitive  strategies,  attitude, 
and  motor  skills.  Establishing  the  type  of  learning  outcome  in  relation  to  the  goal  will  assist  the  OELE  designer  in 
creating  a foundation  for  guiding  the  selection  of  strategies  later  in  the  design  process. 

Instructional  content.  When  determining  the  instructional  content,  the  designer  may  encounter  pre- 
determined expectations  related  to  the  learning  environment.  In  this  instance,  it  may  become  a matter  of  elaborating 
upon  or  refining  what  has  been  covered  in  the  learning  environment  in  prior  implementations.  This  part  of  the 
design  phase  is  important  as  it  enables  the  designer  to  focus  later  efforts,  especially  in  terms  of  development.  It 
should  he  noted  that  like  the  establishment  of  objectives,  who  determines  the  instructional  content  is  very  dependent 
upon  the  roles  established  in  analysis. 

Means  of  instruction.  Means  of  instruction  relates  to  the  instructional  strategics  and  media  cmplo>ed  in  the 
OELE  (West,  Farmer,  & Wolff,  1991).  This  sub-component  of  the  design  phase  can  be  broken-down  into  three 
subsequent  areas. 

1.  How  the  content/processes  are  best  taught/learned.  Traditionally,  instructional  designers  have 
looked  to  specific  strategies  based  on  learning  outcome  to  guide  how  to  teaeh/assist  learners  in 
learning  content  and  processes.  As  stated  by  Jonassen  (1997),  “...the  most  pr.vasivc  assumption  of 
instructional  design  is  that  different  learning  outcomes  necessitate  different  conditions  of  learning 
(Gagne,  1966)’’  (p.  66).  Books  have  been  written  devoted  to  the  topic  of  instructional  strategies  (see, 
for  example,  Gagne,  1985;  Gagne  & Medskar,  1996;  Leshin,  Pollock,  & Reigcluth,  1992)  and  software 
programs  have  been  developed  to  assist  designers  in  creating  instruction  based  on  specific  outcomes 
and  strategics  (see,  for  example,  Designer's  Edge). 

Given  the  emphasis  on  problem-solving  and  critical  thinking  in  OELEs,  how  the  contcnl/processes 
arc  best  taught/lcarncd  takes  on  an  additional  level  of  challenge  when  the  environment  is  open-ended 
in  its  orientation.  “Because  problem-solving  outcomes  arc  not  sufficiently  acknowledged  or 
articulated  in  the  instructional-design  literature,  little  advice  about  how  to  design  problem-solving 
instruction  is  available”  (Jonassen,  1997).  Jonassen  (1997)  has  proposed  two  instructional  design 
models  for  the  design  of  well-structured  and  ill-structured  problems.  His  ill-structured  problems 
instructional  design  model  holds  particular  promise  for  OELEs;  however,  this  is  an  area  in  need  of 
furthci  icscarch  and  exploration. 
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2.  How  to  best  teach/learn  cognitive  strategies.  The  development  of  cognitive  strategies  is  a critical 
process  in  OELEs.  Assisting  and  encouraging  learners  in  the  development  and  lefincmcnt  of  cognitive 
strategies  should  be  a critical  part  of  what  occurs  in  an  OELE.  This  is  an  instructional  strategy  area 
which  has  been  developed  by  Gagne’  (1985).  Following  the  basic  tenets  of  his  specified  conditions  for 
learning  can  greatly  assist  the  learner  as  they  engage  in  an  OELE.  As  summarized  by  Driscoll  (1994), 
these  conditions  include: 

• describe  or  demonstrate  the  strategy; 

• provide  a variety  of  occasions  for  practice  using  the  strategy;  and 

• provide  informative  feedback  as  to  the  creativity  or  originality  of  the  strategy  or  outcome 
(p.  345). 

3.  Selection  of  the  delivery  media.  In  an  OELE,  the  delivery  media  are  often  many  and  varied. 
Ranging  from  books,  to  people,  to  on-line  databases,  an  underlying  assumption  in  OELEs  is  the 
provision  of  multiple  resources  to  help  facilitate  the  development  of  multiple  perspectives. 

Media  should  not  only  vary  in  terms  of  the  physical  mode,  but  the  content  delivered  via  the  media 
should  also  vary.  This  will  also  help  facilitate  the  creation  of  multiple  perspectives  as  the  learner  seeks 
to  make  meaning  and  establish  understanding. 

Evaluation  methods.  Evaluation  is  represented  as  an  underlying  process  in  the  model  depicted  in  Figure  I. 
The  selection  of  evaluation  methods  as  a part  of  the  design  process  focuses  on  assessment  methodologies  for  the 
instruction,  as  well  as  learner  achievement.  As  with  other  components  discussed  in  the  design  phase,  evaluation 
methods  are  dependent  upon  the  participant  roles  defined  in  the  analysis  phase  of  the  instructional  design  process.  If 
the  learner  is  taking  a lead  role  in  the  environment,  evaluation  may  be  focused  solely  upon  self-critique  measures.  If 
the  learner  and  educator  are  working  on  equal  terms  in  the  OELE,  evaluation  methods  may  be  determined  by  both 
parties. 

A critical  assumption  in  an  OELE  involves  learner  engagement  in  self-monitoring  and  reflection  as  a part 
of  involvement  in  the  environment.  This  should  be  taken  into  consideration  when  creating  evaluation  methods. 


Development  and  Implementation 

Thc  development  phase,  like  the  design  phase,  overlaps  with  several  other  areas  in  the  design  process, 
including  design  and  implementation.  As  depicted  in  Figure  1,  development  and  implementation  almost  work 
“hand-in-hand”  in  terms  of  creating  the  OELE.  This  overlap  makes  it  logical  to  link  these  two  phases  of  the  process 
together  in  discussion.  The  ways  in  which  the  model  is  depicted  also  reflects  essential  clement  in  OELE  design: 
development  is  on-going,  almost  throughout  thc  duration  of  thc  course. 

Development  and  implementation  place  the  designer  in  “doing”  mode.  At  this  point  in  the  design  of  thc 
OELE,  thc  designer  focuses  on  thc  creation  of  thc  various  resources,  and  potentially  tools,  that  the  learner  will  need 
to  fully  engage  the  environment.  Special  considerations  arc  discussed  below. 

Resources.  Resources  are  a critical  component  in  OELEs.  In  many  instances,  they  serve  as  thc  stepping 
stones  for  thc  building  of  understanding  (Hannafin  ct  al.,  1994).  As  such,  OELE  resources  are  not  limited  in  terms 
of  either  their  format  or  function.  OELE  resources  rnay  include  people  such  as  thc  educator,  fellow  learners,  or 
external  experts.  Other  resources  utilized  in  an  OELE  may  he  electronically  mediated  in  some  fashion  (e.g.,  video, 
Web  sites,  sound  files)  or  be  print-based  in  their  orientation  (e.g.,  books,  journal  articles,  pamphlets).  Still  other 
OELE  resources  may  be  external  agencies  which  store  vast  amount  of  information,  such  as  academic  or  public 
libraries. 

The  function  of  OELE  resources  is  just  as  varied  as  thc  format*  in  which  they  arc  delivered.  Thc  resources 
(whether  selected  by  the  learner  or  the  educator)  may  serve  to  displace  myths  related  to  a particular  topic.  OELE 
resources  may  also  act  as  factual  information  providers.  Depending  upon  how  it  is  used,  the  resource  may  also 
provide  an  incentive  for  the  learner  to  look  at  an  issuc/lopic  from  an  orientation  they  never  considered. 

While  providing  resources  in  a variety  of  formats  is  an  important  consideration  in  OELEs,  taking  into 
account  the  message  being  conveyed  by  thc  resource  as  they  arc  used  within  the  implementation  of  the  OELE  is  also 
critical.  In  seeking  to  expand  Ihc  learner’s  orientation  to  multiple  perspectives  (a  key  characteristic  of  OELEs), 
OELEs  demand  that  the  resources  selected  represent  varied  orientations.  While  varied,  they  must  also  be  “bound” 
logelhcr  functionally  in  the  environment  in  ways  that  converge  upon  a theme,  yet  diverge  to  support  unique 
purposes  and  goals. 
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Another  critical  component  of  OELE  resources  relates  to  the  actual  creation  of  the  resources.  OELE 
resources  arc  not  limited  to  those  which  arc  developed  in-house.  Perhaps  one  of  the  biggest  challenges  associated 
with  the  design  of  OELEs  comes  in  the  gathering  of  the  resources  for  the  environment.  While  in  some  instances  this 
may  involve  first-hand  creation,  often  it  involves  seeking  and  gathering  the  resources  into  a readily  accessible  space 
so  multiple  learners  have  ready  access  to  the  necessary  information. 

On-going  effort.  A second  element  in  the  development  and  implementation  phases,  on-going  effort,  serves 
to  illustrate  the  strong  tic  between  the  two  areas.  Certainly,  implementation  is  an  on-going  effort  throughout  the  life 
of  the  OELE.  However,  development  work  can  be,  and  often  is,  an  on-going  effort.  The  types  of  resources  needed 
will  likely  change  as  the  problems  posed  by  cither  the  educator  or  learner  emerge  and  grow.  Continuous 
examination  and  evaluation  of  the  resource-base  is  essential  for  assisting  learners  in  their  evolving  understanding. 

Evaluation 

Evaluation  is  the  process  used  to  provide  feedback  to  designers,  enabling  continual  improvement  of  the 
OELE  (Ken,  <orrison,  & Ross,  1998).  In  an  OELE,  evaluation  is  a continuous  process.  Starling  during  the 
analysis  phase  of  OELE  design,  evaluation  underlies  all  other  phases  and  processes  associated  with  the  creation  of 
an  OELE.  Perhaps  what  is  more  critical  is  that  the  evaluation  is  engaged  by  all  participants  in  the  GELT.  and  in  a 
variety  of  ways. 

The  designer  engages  in  evaluation  as  a self-check  to  ensure  that  they  have  considered  all  asp,  » in  the 
creation  of  the  OELE,  and  to  measure  effectiveness  of  instruction,  as  well  as  the  learning  environment  itself.  The 
learner  engages  in  self-evaluation  and  re-assessment  throughout  their  involvement  in  the  OELE.  By  cutting  across 
the  design  process,  the  evaluation  phase  brings  together  a variety  of  audiences  and  perspectives,  strengthening  the 
further  enhancement  of  the  OELE. 

Maintenance 

Maintenance  involves  the  continual  upkeep  of  the  OELE.  Tessmer  (1991)  refers  to  this  as  sustaining  use, 
and  recommends  that  strategies  for  maintenance  begin  during  the  analysis  phase.  Maintenance  is  much  like 
evaluation  — a continuous  process  throughout  the  life  of  the  OELE  (see  Figure  1).  While  often  discussed  in  terms  of 
the  long-term  implementation  of  an  OELE,  maintenance  could  also  he  viewed  as  a process  much  like  evaluation: 
ever-present  and  pervasive  in  OELE  design. 

OELEs  arc  intensive  yet  delicate  environments.  As  such,  like  the  prized  orchid,  they  require  continual 
monitoring  and  care.  During  implementation,  for  instance,  the  instructor  must  be  able  to  discern  if  the  learner  needs 
additional  scaffolding,  is  running  into  “dead  ends,”  or  is  using  incompatible  approaches  or  strategics.  The  learner 
holds  primary  responsibility  for  the  learning  process,  yet  the  instructor  or  designer  is  also  responsible  for  creating  a 
“space”  for  problem-solving  to  occur.  Through  continual  monitoring,  the  OELE  can  sustain  growth  in 
understanding. 

Guidelines  for  Using  the  OELE  Design  Model 

The  model  discussed  in  the  previous  section  was  generated  based  on  key  characteristics  and  assumptions  of 
OELEs,  as'  well  as  our  work  in  the  development  of  OELEs.  The  model  does  not  yet  have  a strong  empirical 
foundation  to  support  it,  however,  it  has  proven  to  be  an  effective  model  for  us  in  the  design  of  our  own  OELEs.  As 
caveats,  we  offer  the  following  guidelines  for  use  of  the  model: 

• The  model  is  descriptive  in  nature.  It  was  not  created  to  prescribe  specific  steps  for  the  design 
of  OELEs. 

• The  model  has  been  used  for  the  development  of  courses  following  the  theoretical  foundation 
of  open-ended  icarning. 

• While  the  model  has  been  informally  tested  in  facc-to-facc  and  distance  formats,  other  OELE 
implementations  remain  unexplored. 

Conclusions  and  Implications 

The  proposed  model  for  the  design  ot  OELE  courses  is  extensive  in  scope,  hut  far  from  conclusive.  The 
model  discussed  in  this  paper  is  hased  on  our  own  experiences  in  the  creation  of  OELE  courses,  both  in  face-to-face 
and  distance  formats.  There  exists  (he  probability  that  many  elements  critical  to  the  design  of  OELEs  have  been 
overlooked  in  the  discussion.  What  is  clear  is  that  more  empirical  work  is  needed  ;o  understand  if  the  proposed 


176 


ion 


model  is  an  accurate  reflection  of  how  to  effectively  and  efficiently  design  OELEs.  However,  the  model  does 
provide  a starling  point  for  further  exploration  and  discussion. 
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Abstract 

This  study  reports  on  the  formulation  of  educational  technology  policy  in  three  Illinois  Unit  (K~I2)  School 
Districts.  The  major  findings  of  the  study  include:  (a).  Educational  technology  policy  formulation  focused  on 
collecting  the  objects  of  technology , such  as  computers,  modems,  networks,  and  the  like,  rather  than  viewing 
educational  technology  as  a systematic  process  of  achieving  goals,  (h).  Active  leadership  from  a superintendent 
essential  in  each  school  district,  (c)  Formulation  of  the  plans  was  more  than  an  empowered  committee  or 
executive  blessing.  It  required  active  participation  by  a superintendent.  Other  discussion  includes  a planning 
template  was  developed  as  insight  developed  from  the  study. 

School  districts  are  seeking  ways  to  incorporate  the  latest  trends  of  hardware  and  software  into  their 
classrooms.  Frequently,  the  proponents  who  promote  the  attributes  of  hardware  in  classroom  promise  prosperity, 
school  restructuring,  and  reformation  of  schools  (Fiskc,  199V,  Rcigeluth,  Annelli  & Otton,  1992;  Stanzione  & 
Thompson,  1993);  improved  attendance  (Rcigeluth  et  al.,  1992);  increased  learning  gains  (Doyle  & Levinson, 
1993b;  Fiske,  1993;  Gates,  1993;  Guthrie,  Garms,  &.  Pierce,  1988);  improved  motivation  (Kcndl  & Liberman,  1988; 
Reigeluth  et  al.,  1992);  development  of  higher  order  thinking  skills  (Rockman,  1993);  higher  teacher  and 
administrator  productivity  (Guthrie  et  ah,  1988;  Doyle  & Levinson,  1993a,  1993b);  and  the  "[relief  of]  teachers  from 
some  of  the  more  time-consuming  and  inefficient  aspects  of  their  traditional  role  (Schlechty,  1991,  p.  75). 

Frequently,  technology  proponents  and  enthusiasts  view  technology  as  hardware  only,  and  frequently 
microcomputers  and  their  peripherals.  However,  the  educational  technology  community  views  the  subject  as  a 
process,  such  as  the  one  these  writers  frequently  use.  "Educational  technology  is  the  systematic  means  of  achieving 
stated  goals."  The  process  includes  people,  instructional  design  models,  organizational  and  management  theories 
and  practices,  research  and  evaluation  techniques,  and  equipment  and  materials.  For  the  purposes  of  this  report,  the 
terms  "technology"  and  "educational  technology"  will  refer  to  the  process.  When  objects,  such  as  microcomputers 
and  their  peripherals,  are  points  of  discussion,  generally  they  will  be  referred  to  as  "hardware." 

Prescriptive  literature  describes  how  school  districts  should  plan  to  bring  hardware  and  software  into  their 
districts  (Barkman,  1989;  Burrus,  1993;  Crow  & Rariden,  1993;  Dormant,  1992;  Doyle  & Levinson,  1993a,  b;  Dyrili 
& Kinnaman,  1994a,  b;  Kinnaman,  1991;  Missouri  School  Boards  Association,  no  date;  See,  1992).  However,  little 
studies  exist  to  document  the  actual  practice,  the  subject  of  this  report. 

The  Study 

During  late  1994  and  early  1995,  the  principal  investigator  interviewed  36  human  subjects  from  three 
Illinois  Unit  (K-J2)  School  districts  to  investigate  the  nature  of  technology  planning  in  their  district.  The  subjects 
were  from  the  districts’  planning  committees,  such  as  the  superintendent,  central  office  administrators,  principals, 
technology  coordinators  and  teachers,  and  others  who  influenced  the  planning  process.  Participating  school  districts 
were  selected  from  a population  of  nearly  50  Unit  districts.  The  researchers  chose  to  work  with  districts  that  varied 
in  demographics.  The  study  took  its  foundation  from  qualitative  research  to  develop  case  studies  to  answer  the 
research  question  based  on  Hubcrman  and  Miles  (1994),  Marshall  and  Rossman  (1989),  Mcrriam  (1991),  Stake 
(1994)  and  Yin  (1994),  The  data  for  the  study  were  gathered  through  interviews  with  technology  planners  and  others 
who  affected  the  planning  process  and  inspection  of  relevant  documents  to  triangulate  on  the  facts  of  each  case. 
Interview  questions  were  developed  from  policy  studies  (Spitzcr,  1992;  Quinn,  1992)  and  general  questions  about 
the  participants  views  and  use  of  educational  technology. 
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The  questions  were  formulated  and  used  to  develop  rapport  with  the  participants  and  to  determine  reasons 
for  hardware  and  software  usage,  as  well  as  to  help  develop  insights  that  arc  explored  throughout  the  case  reports 
(Hunt,  1995).  The  questions  were: 

1.  Tell  me  about  yourself,  your  background,  and  your  current  position  with  this  school  district. 

2.  How  do  you  define  educational  technology? 

3.  How  would  you  like  to  see  emerging  equipment  like  multimedia  computers,  CD-ROMs,  laser  discs,  the 

information  super  highway,  and  the  like  used  in  your  school  district? 

4.  What  differences  do  you  sec  in  the  current  equipment  just  described  as  compared  to  traditional  audio 

visual  equipment,  such  as  overhead  and  film  strip  projectors,  and  audio  tape  players? 

5.  What  arc  your  sources  within  the  district  for  information  about  technology? 

6.  What  are  your  sources  outside  the  district  for  technology  information? 

7.  Do  you  make  an  effort  to  look  for  technology  developments  from  these  external  sources,  or  is  the 

technology  news  secondary? 

3.  How  do  you  (individually)  use  technology  at  work  and  at  home? 

The  first  question  was  asked  to  learn  about  the  educational  background  and  educational  experiences  of  the 
participants.  The  second  question  was  developed  because  of  many  informal  discussions  with  interested  educators, 
courscwork  in  the  researcher's  curriculum,  participation  in  the  educational  technology  planning  process  in  his  own 
school  district,  and  reading  about  various  definitions  of  technology  and  educational  technology.  The  researcher  was 
curious  about  how  policy  planners  defined  the  term  educational  technology  and  whether  their  definition  itself  might 
impact  how  they  developed  technology  policy. 

The  popular  press  repeals  the  promises  that  technology  will  make  our  lives  better.  The  third  and  fourth 
questions  were  used  to  probe  participants'  perceptions  of  the  general  promises  - collected  through  informal 
discussions  and  reading  — that  newer  equipment  would  make  students  better  learners  and  schools  better  places  than 
had  happened  with  traditional  audiovisual  equipment  and  other  educational  initiatives.  It  was  the  researcher's  intent 
to  probe  the  participants'  reasons  for  the  use  of  technology.  From  his  own  reading  and  discussions  with  educators 
and  other  technologists,  the  researcher  hypothesized  that  technology  planners  would  focus  on  hardware,  technology 
promises,  and  planning  literature  that  had  no  research  foundation.  Did  they  repeat  and  apply  the  mass-marketed 
technology  promises  in  their  own  school  districts  or  challenge  and  examine  them?  Did  planning  committees  use 
prescriptive  literature  to  develop  their  technology  policies?  The  researcher  hypothesized  that  this  study  would 
answer  these  questions  in  the  affirmative. 

Questions  five,  six,  and  seven  were  develop  to  investigate  the  sources  of  technology  information  that 
policy  planners  used  to  make  their  decisions.  As  statcu  above,  much  of  the  information  flow'  about  technology  is  not 
research-based.  Is  this  raw,  opinion-based,  unfiltered,  mass -market  information  reaching  the  policy  planners?  If  so, 
how  do  they  react  to  it? 

The  final  question  in  this  series  was  asked  to  seek  the  level  of  hardware  and  software  use  by  the 
participants.  Were  the  participants  users  of  hardware  and  software  or  were  they  non-users  with  opinions  about  how 
hardware  and  software  should  be  used? 

To  investigate  the  nature  of  policy  planning,  Spilzer’s  (1992)  and  Quinn's  (1992)  questions  for  ~cd  the  basis 
of  further  inquiry.  Spitzer,  writing  from  an  HPT  perspective,  offered  ways  how  to  design  and  study  effective 
interventions  in  the  business  community.  Quinn  designed  a major  evaluation  study  for  the  Chicago  Public  Schools. 
Quinn’s  questions  appear  to  be  similar  to  those  of  Guba  and  Lincoln  (1981).  These  sources  provide  the  foundational 
questions  for  a complete  evaluation  study,  but  they  were  molded  together  and  ap[  lied  to  this  study  to  investigate  the 
planning  process  in  detail.  The  questions  wcie: 

1.  What  is  the  need  for  the  technology  policy? 

2.  Who  is  the  sponsor  (champion,  originator/? 

3.  What  arc  the  expectations  and  commitment  of  the  sponsor? 

4.  What  other  key  decision  makers  in  the  organization  have  an  interest  in  this  policy? 

5.  What  arc  their  expectations? 

6.  What  is  the  scope  of  the  policy? 

7.  What  arc  the  goals  of  the  policy? 

8.  What  documents  have  been  supplied  toothers  in  the  organization  to  implement  the  plan? 

9.  What  is  the  time  frame  of  the  project? 


1 0.  What  other  individuals  or  groups  may  he  affected  by  the  technology  policy?  What  is  the  impact  of 
the  policy?  Did  they  have  any  input  on  this  policy?  If  so,  in  what  ways? 

1 I . What  are  their  expectations? 

1 2.  What  related  policies  or  plans  have  been  implemented  in  the  organization? 

1 3.  What  were  the  results? 

1 4.  What  sources  of  support  exist  for  the  policy? 

1 5.  What  are  the  constraints  on  the  formulation  of  this  policy? 

1 6.  What  other  economic,  political,  or  cultural  factors  affected  the  formulation  of  this  policy? 

1 7.  What  individual  ideas  do  you  have  about  the  formulation  of  this  technology  plan? 

These  questions  were  investigated  through  the  dialog  of  scmislructurcd  interviews  with  the  primary 
participants  and  by  inspecting  the  technology  policy  and  related  school  district  documents.  These  interactions  raised 
new  questions,  which  led  to  follow-up  inquiries  with  some  participants.  Furthermore,  interviews  with  primary 
participants  revealed  issues  that  generated  interviews  with  additional  individuals  about  specific  topics.  Normally 
those  individuals  did  not  experience  the  full  interview  process;  rather,  the  interview  focused  on  spccificdelails. 

For  example,  in  one  participating  school  district,  teachers  were  encouraged  to  write  grants.  During  one 
interview',  a primary  participant  mentioned  how  two  teachers  in  her  building  wrote  a technology-related  grant. 
Those  teachers  were  then  interviewed  about  their  grant-writing  ideas  and  the  culture  of  grant  writing  in  the  district. 

Furthermore,  at  the  beginning  of  the  1994-95  school  year,  .v  principals  in  one  district  influenced  their 
district's  planning  process.  They  became  pari  of  the  planning  committee,  but  just  before  its  final  revision.  These 
individuals  were  asked  for  their  input  since  they  had  been  previously  employed  by  school  districts  that  bad  made 
considerable  use  of  hardware  and  software.  They  raised  questions  about  why  a certain  computer  platform  was 
chosen  and  why  computer  networks  were  not  part  of  the  plan.  This  led  the  planning  committee  to  reinvestigate  and 
explain  its  platform  decision  and  to  investigate  networks.  It  was  necessary  to  interview  these  newcomers,  even 
though  their  participation  had  been  limited. 

In  another  district,  a former  superinlc  ident's  influence  .showed  in  the  plan,  making  it  vital  to  interview  him. 
His  activities  while  superintendent  of  that  school  district  had  long-lasting  impacts  on  its  technology  culture. 

In  two  cases,  additional  individuals  were  interviewed  about  specific  issues  to  corroborate  what  primary 
participants  said.  These  individuals  became  secondary  participants  because  they  had  little  or  no  knowledge  of  the 
planning  function,  but  they  had  influenced  or  been  influenced  by  the  process. 

For  this  report,  the  participating  school  districts'  names  have  been  changed  because  of  the  promise  of 
anonymity  by  the  researchers.  Furthermore,  pseudonyms  were  not  developed  for  participants.  Rather,  they  arc 
identified  by  their  job  titles. 

The  River  View  Technology  Policy 

At  the  time  of  this  study,  River  View  School  District  was  comprised  of  six  schools,  enrolling  about  4.500 
students.  The  population  was  largely  White  (91%)  and  the  attendance  rate  was  high  (95%),  as  stated  on  the  School 
Report  Card.  The  district's  single  high  school  graduated  89%  of  the  students  who  enrolled  as  freshmen.  The  faculty 
was  almost  all  White  (a  School  Report  Card  category),  very  experienced,  and  the  average  teacher  was  paid  nearly 
$2,500  above  the  state  average.  The  school  district  spent  about  $1 ,000  per  pupil  less  than  the  state  average. 

The  context  of  technology  planning  and  the  school  climate  in  general  was  flavored  by  a major  funding 
issue  and  political  fallout  related  to  previous  shifting  of  money  between  funds  in  the  school  district's  budget  hecausc 
of  failed  referenda.  This  resulted  in  a slow  drain  of  financial  resources  from  activities  that  directly  affected  students 
in  classrooms,  such  as  acquisition  of  hardware  and  software,  hampered  technology  planning,  and  negatively 
influenced  school  climate,  in  general. 

Participants  in  this  study  ('Superintendent.  Assistant  Superintendent  Curriculum, Technology  Coordinator, 
four  teachers,  former  superintendent,  two  principals,  and  director  of  (he  public  library)  were  the  most-spirited 
among  the  three  districts  studied.  They  spoke  passionately  about  bringing  technology  into  their  school  district.  Some 
of  the  teachers  had  strong  characters,  and  they  did  not  seem  to  accept  leadership  well.  This  appeared  to  cause  chaos 
at  times  in  technology  committees  activities,  and  a disjointed  educational  technology  policy  resulted,  rather  than  a 
cohesive,  insightful  policy. 

At  the  time  of  this  stud} , educational  technology  was  being  considered  again  after  a period  of  dormancy.  A 
former  superintendent  had  placed  computers  in  the  main  office  and  into  classrooms.  He  believed  that  students 
should  use  computers  to  learn.  However,  his  initiative  was  sidetracked  by  the  financial  and  public  relations  prohlems 


in  the  school  district,  which  eventually  led  to  his  resignation.  Following  his  departure,  the  district  had  two 
superintendents  in  two  years,  which  put  the  technology  initiative  into  a dormant  state.  At  the  time  of  the  study,  the 
district  was  still  recovering  from  its  financial  problems,  which  had  prohibited  the  district  from  updating  equipment 
and  implementing  the  plan.  Teacher  participants  appeared  cautiously  optimistic  that  the  plan  would  be  implemented 
at  some  future  date  when  the  district  could  afford  to  fund  it. 

Technology  planning  in  River  View  took  place  in  three  areas:  formal  planning  through  committee,  informal 
communication  caused  by  personal  agendas,  and  independent  action  in  each  school  that  is  funded  outside  the  review 
process.  The  district's  previous  financial  problems  and  changing  two  superintendents  within  two  years  affected  the 
district's  progress  in  many  areas,  especially  in  technology  and  its  proposed  implementation. 

The  previous  superintendent’s  initial  actions  toward  a district-wide  technology  initiative  in  the  mid-1980s 
continued  to  affect  the  district.  He  purchased  computers  that  were  discontinued  models  at  the  time  of  the  purchase, 
and  they  are  still  used  in  the  elementary  schools.  His  vision  of  technology  applications  continued  to  get  mixed 
reviews  by  veteran  teachers.  His  positive  attitude  toward  technology  was  transferred  to  his  technology  coordinator 
who  was  the  sponsor  of  the  written  plan  discussed  in  this  chapter. 

Planners  were  concerned  about  purchasing  up-to-date  equipment,  maintaining  it,  and  purchasing 
replacement  equipment.  Other  planning  issues  were  parity  and  equity,  training,  and  adoption  of  a platform,  which 
itself  was  a small  issue,  but  its  discussion  required  considerable  time. 

In  the  informal  mode,  decisions  and  purchases  were  made  and  donations  were  received  without  formal 
review  of  the  committee  or  guidance  from  an  established  procedure,  The  local  public  library  donated  modems  to  the 
schools  so  students  could  view  the  library's  holdings  via  computer.  In  addition,  principals  attempted  to  secure 
hardware  and  furniture  through  building-wide  fund  raisers,  while  individuals  influenced  the  expenditure  of  Chapter 
1 funds  without  consultation  of  the  committee. 

The  plan  was  constrained  by  the  financial  state  of  the  school  district,  politics,  the  participant's  views  of  the 
administration's  commitment  to  technology,  definitions  of  educational  technology,  and  communication  of  the 
committee  with  its  community. 

The  Orchard  Heights  Technology  Policy 

At  the  lime  of  this  study,  Orchard  Heights  School  District  was  comprised  of  15  schools,  enrolling  about 
8,500  students.  The  population  was  60%  White,  19%  Black,  and  20%  Hispanic;  it  had  a 93%  attendance  rale.  The 
high  school  graduated  82%  of  the  seniors  who  enrolled  as  freshman.  The  faculty  was  largely  White  (91%),  very 
experienced,  and  the  average  teacher  was  paid  nearly  $2,000  above  the  state  average.  The  school  district  spent  about 
$1 ,000  per  pupil  less  than  the  stale  average. 

The  participants  in  this  study  (Superintendent.  Technology  Coordinator,  four  principals,  nine  teachers,  and 
Mayor's  assistant)  expressed  commitment  to  technology  applications  in  their  school  district.  Although  the 
researcher  did  not  see  the  lively  spirit  as  in  the  previous  school  district,  the  participants  spoke  with  conviction  of 
their  ideas  and  trust  in  the  technology  planning  process  and  the  potential  of  technology  in  their  school  district. 

During  the  study  the  school  district  passed  a referendum  to  finance  its  technology  initiative.  The 
participants  were  convinced  that  the  district  did  not  have  financial  resources  to  fund  a technology  initiative  because 
the  state  had  placed  a cap  on  annual  property  tax  increases  in  the  school  district’s  county  at  a 5%  increase  or  the 
actual  cost  of  living  increase,  whichever  was  lower.  The  voters  dispelled  that  concern. 

A core  of  technology  planners  - led  by  the  superintendent,  tcchrology  coordinator,  ind  technology 
committee  co-chair  --  formed  a nucleus  of  policy  makers  in  the  district.  However,  some  planners  identified  other 
influential  faculty  members  as  important  players  in  the  plan's  formulation. 

Before  the  formulation  of  the  plan  that  was  the  subject  of  this  study,  the  district's  efforts  in  technology 
planning  and  implementation  were  irregular.  Planning  and  equipment  acquisitions  were  made  by  a district  computer 
committee  in  various  forms.  Three  grants  from  the  regional  telephone  company  created  disparity  because  it  focused 
only  three  of  (he  district's  schools 

The  plan  that  was  investigated  in  this  study  contained  philosophical  and  visionary  statements,  student  goals 
and  hardware  requests.  Essentially  the  plan  attempted  to  level  individual  and  school  initiatives  that  were  encouraged 
by  administration  because  the  school  district  did  not  provide  financing  fora  consistent,  district-wide  initiative. 

The  views  of  the  technology,  such  as  technology  is  an  enhancement,  affected  the  formulation  of  the  plan. 
Participants  appeared  to  view  computer-based  technologies  in  differently  from  traditional  equipment  like  movie  and 
slide  projectors. 
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The  committee  that  formulated  the  plan  was  composed  of  teachers  from  schools  and  administrators.  The 
committee  focused  on  acquiring  equipment  to  enhance  education.  Other  planning  issues  the  committee  considered 
were  parity,  networking,  and  staff  development. 

The  high  school  principal  appeared  to  influence  the  contents  of  the  final  plan  as  he  sent  detailed  letters  to 
the  planning  committee's  sponsors.  He  was  successful  in  gelling  networks  and  mobile  computer  labs  for  his  school 
into  the  plan.  The  principal's  influence  was  noted  by  participant's  in  all  levels  of  the  planning  committee. 

The  planning  process  was  constrained  by  the  district's  lack  of  defined  leadership  because  no  individual  or 
group  appeared  to  want  to  take  the  lead  on  planning  issues.  The  long-term  culture  of  freedom  produced  pockets  of 
hardware  that  the  plan  attempted  to  acquire  and  distribute  to  all  schools.  Politics,  especially  those  associated  with  the 
principal's  push  to  make  his  school  the  technology  showcase,  constrained  formulation.  The  plan  was  limited  by  the 
committee's  perceived  view  of  community  commitment.  They  stated  that  the  they  thought  the  community  would  not 
fully  support  a technology  initiative  to  provide  full  technology  opportunities  for  children.  In  addition,  the  plan  was 
limited  by  the  lack  of  communication  with  the  school  community. 

During  this  study,  the  school  district  was  working  with  the  city  to  establish  a city-wide  network  with  other 
public  entities  and  private  university  in  the  city.  This  network  was  to  have  been  the  backbone  for  large-group 
learning  in  the  Orchard  Heights  schools. 

The  State  Park  Technology  Plan 

At  the  time  of  this  study,  State  Park  School  District  was  comprised  of  14  schools,  enrolling  about  5,500 
students.  The  student  population  was  the  most  diverse  of  the  three  school  districts  in  this  study.  Thirty-seven  percent 
of  students  were  White,  57%  were  Black,  and  5%  were  Hispanic.  Over  half  of  the  students  were  low  income,  but  the 
attendance  rate  was  92%.  The  high  school  graduated  only  67%  of  the  seniors  who  enrolled  as  freshmen.  The  faculty 
was  83%  White,  16%  Black,  and  1%  Hispanic;  averaged  nearly  15  years  of  experience;  and  were  paid  about  $1,000 
bci^w  the  state  average.  The  school  district  spent  about  $100  per  pupil  less  than  the  state  average.  The  current 
superintendent  noted  that  one  of  the  district's  main  problems  was  that  the  district  had  a high  mobility  rate,  in  that 
.bout  one-third  of  the  students  in  the  school  district  either  enrolled  or  left  the  district  during  each  school  year.  The 
Assistant  Superintendent  for  Business  Affairs  related  that  the  district  was  poor  financially  and  that  it  received  most 
of  its  operating  revenue  from  state  (50%)  and  federal  (17%)  sources. 

Before  the  current  plan,  the  district's  technology  initiatives  were  limited  to  labs  in  each  attendance  center. 
The  plan  that  was  the  subject  of  this  study  was  sponsored  by  a superintendent  who,  as  a new  employee  in  the  school 
district,  brought  high  energy,  new  ideas,  and  a technology  vision  and  experience  to  the  district.  Because  of  his  task 
of  reorganizing  and  rc-encrgizing  the  school  district,  he  turned  to  an  outside  consultant  - IBM  Corporation  - for 
assistance.  An  IBM  facilitator  led  a district  planning  committee  through  a two-day  planning  session  based  on  an 
IBM  process,  known  as  a mctaplan,  to  develop  the  plan.  Before  the  actual  planning  session,  the  planning  committee 
visited  schools  that  were  thought  to  be  in  advanced  stages  of  technology  implementation;  counted  the  computers  in 
their  own  district;  and  surveyed  their  distrir  teachers  on  computers,  although  it  is  unknown  if  the  survey  results 
actually  were  used  during  planning.  During  the  session,  the  planners  developed  a mission  statement,  identified 
problems  and  solutions,  and  proposed  an  action  plan  in  a dynamic  and  positive  group  interaction.  The  plan  was 
constrained  by  the  accelerated  planning  process  and  communication  with  the  school  community. 

The  plan  was  developed  in  1992  and  essentially  placed  on  the  shelf,  because  of  funding.  Planners  returned 
to  their  regular  duties,  and  the  sponsor  retired  from  the  school  district  and  education  on  June  30,  1993. 

Cross-Case  Comparison 

In  all  three  districts,  nearly  every  participant  gave  definitions  or  descriptions  of  educational  technology  that 
were  related  to  what  Sacttlcr  (1990)  called  the  "physical  science  approach"  (p.  5). 

Participants  in  River  View  called  technology  "a  tool"  or  gave  examples  of  hardware  and  software.  In 
Orchard  Heights,  the  "enhancement"  view  was  similar  to  River  View's  tool  description.  Examples  of  hardware  and 
software  were  given  in  State  Park.  The  Orchard  Heights  superintendent  was  the  only  participant  to  give  a process- 
oriented  definition: 

It's  far  more  than  the  hardware  and  software.  I believe  that  it's  almost  the  process  of  exploration. 

Technology  is  really  a step-by-step  process  of  problem  solving.  . . . It’s  not  necessarily  what  it  is,  but  what 

it  can  do. 

Furthermore  the  following  points  can  be  extracted  from  the  study. 


J . Educational  technology  policy  formulation  focused  on  collecting  the  objects  of  technology,  such  as  computers, 
modems,  networks,  and  the  like,  rather  than  viewing  educational  technology  as  a systematic  process  of 
achieving  goals. 

2.  Active  leadership  from  a superintendent  was  essential  in  each  school  district.  Formulation  of  the  plans  was 
more  than  an  empowered  committee  or  executive  blessing.  It  required  active  participation  by  a superintendent. 

3.  School  districts  developed  educational  technology  policies  regardless  of  their  financial  state. 

4.  Educational  technology  policy  formulation  occurred  without  regard  for  student  demographics. 

5.  Applied  technology  or  technology  education  --  including  electronics,  robotics,  video  production,  industrial 
technology  and  metals  technology  - was  part  of  educational  technology  policy  formulation  in  two  of  the  three 
school  districts. 

6.  While  planning  focused  on  the  objects  of  educational  technology,  planners  took  little  action  on  other  elements 
of  educational  technology  planning,  such  as  staff  development,  finance,  evaluation,  and  school  cultural  issues. 

7.  Technology  planners  did  little  to  communicate  aspects  of  their  educational  technology  plan  to  their  school 
communities. 

8.  Educational  technology  policy  was  a political  process.  Whether  it  was  new  superintendent  pushing  his 
technology  agenda  or  a teacher  influencing  a computer  purchase,  politics  were  part  of  the  process. 

9.  The  planning  committees  were  not  representative  of  the  school  community. 

Technology  Planning  Web 

From  listening  to  the  technology  planners  from  the  three  school  districts  describe  the  lechnology^relatcd 
issues,  examining  their  documents,  and  studying  their  plans,  the  researcher  observed  that  some  of  their  planning 
ideas  had  omissions,  such  as  detailed  plans  for  staff  development,  finances,  evaluation,  and  school  culture  issues. 
From  this  research  he  developed  a planning  template  focused  on  the  planning  aspects  of  educational  technology. 

The  flow  of  a technology  initiative  starts  with  a vision.  A single  goal  or  a scries  of  goals  should  flow  from 
this  vision.  Learning  activities  are  then  developed  and  implemented.  These  events  are  followed  by  an  evaluation, 
either  formative  or  summative.  In  a systems  approach,  evaluation  is  followed  by  a recycling  component  to  revise 
portions  of  the  learning  experience  that  do  not  meet  the  evaluation  criteria.  Figure  1 shows  the  general  flow  of  a 
technology  initiative. 


Figure  J.  Flow  of  a Technology  Initiative 

Using  a systems  approach  anti  adding  the  omissions  observed  in  the  participating  school  districts'  planning 
activities,  the  researcher  proposes  a technology  planning  template  that  would  be  used  at  the  goal  and  development 
steps.  It  is  called  the  "Technology  Planning  V/eb"  and  is  shown  in  Figure  2.  The  components  of  the  web  arc 
evaluation  protocol,  staff  development,  hardware,  finances,  research  and  development,  physical  infrastructure,  and 
polilical/cultural  infrastructure.  Each  component  offers  its  own  challenges  and  systematic  design  solutions.  Most 
important  is  that  each  component  needs  detailed  planning  and  not  just  a mere  mention  in  the  educational  technology 
policy. 
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Figure  2.  Technology  Planning  Web 

The  web  can  be  used  by  a school  district  committee  to  consider  how  to  address  technology  planning.  This 
web  would  be  applied  after  the  planning  committee  has  constructed  a mission,  vision,  or  philosophy  statement. 
Goals  should  be  derivatives  of  the  declarations  of  the  committee's  preferred  future  state  of  the  school  district.  As 
each  technology  goal  is  composed,  the  web  should  be  used  to  remind  the  planning  committee  of  other  issues  that 
exist  for  that  goal.  Each  goal  should  have  its  own  web  so  that  a complete  plan  will  have  a detailed  web  for  each 
goal.  Specific  learning  activities  then  can  flow  from  the  goal  statements,  such  as  those  related  to  keyboarding,  group 
learning,  or  literacy  issues. 

One  omission  that  was  noted  in  the  technology  planning  of  the  three  school  districts  in  the  study  was  that 
no  evaluation  protocol  existed  for  the  plans.  The  participants  offered  various  measurements  of  success:  purchasing 
all  of  the  equipment  in  the  plan,  use  of  the  equipment,  and  integration  into  the  classroom.  However,  these  were  not 
aligned  with  their  preferred  future  states.  Only  a few  participants  mentioned  impacting  the  learning  of  children  as  a 
success  factor. 

Many  instructional  design  models  specify  writing  ihe  evaluation  protocol  at  the  time  the  goal  statement  is 
written  (c.g.  Jonassen,  1988).  One  distinct  advantage  of  writing  the  evaluation  at  that  lime  is  that  the  human  energy 
required  to  reach  consensus  about  a goal  can  be  used  immediately  to  construct  the  evaluation  protocol.  The 
procedure  is  continuous  and  the  evaluation  is  developed  in  the  same  context  as  the  goal  and  not  at  some  later  time  by 
another  committee  that  wonders  about  the  origin  or  reason  for  the  goal  statement. 

For  technology  planning,  Herman  (1994)  suggests  that  evaluations  should  he  based  on  quantitative  and 
qualitative  data.  She  notes  that  educational  technology  initiatives  arc  nearly  impossible  to  evaluate  when  teachers 
arc  encouraged  to  adapt  the  technology  to  their  own  classrooms.  Doyle  and  Levinson  (1993b)  and  Dyrli  and 
Kinnaman  (1994a)  advocate  that  planners  develop  curriculum  activities  that  can  be  uniformly  applied  across  the 
district,  not  adapted  for  individual  classrooms. 

If  a technology  goal  states,  "Students  will  be  engaged  learners,"  the  next  immediate  question  should  he, 
"How  would  we  know  when  we  have  engaged  learners?"  The  committee  could  then  write  the  necessary  protocols 
that  could  be  revisited  at  some  future  date  to  check  on  progress  toward  reaching  that  goal.  At  some  future  date, 
when  the  school  district  writes  ils  annual  report.  "We  arc  a community  of  engaged  learners,"  then  it  could  describe 
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the  justification  for  the  statement.  Once  the  goal  and  evaluation  protocol  are  developed  other  issues  become  evident 
that  are  not  necessarily  linear,  but  are  interrelated  as  in  a web. 

Three  elements  of  the  web  are  inanimate:  hardware,  physical  infrastructure,  and  finances.  Once  the  goal 
and  evaluation  protocol  are  developed,  the  committee  should  make  recommendations  for  hardware,  its  related 
software,  and  necessary  wires  — the  physical  infrastructure  - to  make  it  operate.  The  infrastructure  needed  could  be 
as  basic  as  extra  electrical  outlets  in  a room  to  power  a computer  or  as  sophisticated  as  a district-wide  fiber-optic 
system.  The  Missouri  School  Boards  Association  (no  date)  suggests  that  planners  should  survey  hardware 
possibilities  early  in  technology  planning.  However,  many  instructional  design  models  specify  hardware 
requirements  later  in  the  planning  (c.g.,  Heinich,  Molcnda,  & Russell,  1989).  Hardware  should  be  selected  in 
accordance  with  the  design  models. 

The  hardware  element  in  the  web  should  account  for  obsolete  equipment.  How  will  the  district 
decommission  old  equipment?  Will  equipment  be  placed  in  other  places  in  the  school  or  district  when  it  is  replace 
or  should  it  be  disposed  of  when  it  is  obsolete  for  its  original  purpose? 

Finances  are  another  non-human  element  in  the  web.  Why  develop  a sophisticated  technology  plan  when 
funds  arc  not  available  to  implement  it?  Certainly,  the  sponsor  of  the  State  Park  technology  policy  should  be 
congratulated  for  energizing  the  planning  activity,  but  money  did  not  exist  and  little  hope  existed  that  it  would.  The 
planning  facilitator  stated  that  the  process  she  used  helped  communications  within  the  group  — a powerful  and 
positive  side-effect  of  such  a group  effort  - but  with  the  funding  issues  in  the  school  district,  why  even  start  building 
a plan  with  no  attempt  to  search  for  funds?  An  attempt  to  assure  funds  from  regular  budget  lines,  partnerships,  or 
referenda,  should  be  occurring  at  least  in  parallel  with  a district  planning  process. 

Human  issues  of  politics/culture  and  staff  development  arc  also  key  elements  of  the  web.  How  will  a 
district  address  these  issues?  Burrus  (1993)  stated  that  "technology  changes  everything"  (p.  18).  Whether  the 
district  employs  a sophisticated  planning  process  or  merely  transplants  hardware  in  a library/media  center,  the 
existing  school  culture  is  threatened.  How  will  districts  focus  the  benefits  of  educational  technology  and  restrain 
those  that  may  be  detrimental  to  learning?  Districts  need  to  forecast  these  changes  and  be  attentive  to  them,  if  they 
observe  what  Burrus  claims,  vve  have  no  choice. 

In  the  political  arena,  this  study  noted  that  educational  technology  policy  formulation  is  a political  process. 
The  leadership  of  technology  planning  must  recognize  that  individuals  or  schools  will  attempt  to  influence  the  plan 
to  benefit  the  particular  agenda.  The  leadership  must  be  ready  to  act  to  keep  the  process  focused  on  reaching  a 
school  district  vision  and  be  ready  to  soothe  bruised  egos,  reward  behavior  parallel  to  reaching  district  goals,  and 
prod  lagging  planning  activities. 

Dormant  (1992)  identifies  problems  in  causing  change  in  an  organization.  Culture  and  leadership  are  two 
major  issues  that  cause  or  s,oo  change.  She  quotes  Shein's  definition  of  culture  as  "a  pattern  of  basic  assumptions  -- 
invented,  discovered,  or  developed  by  a given  group  as  it  learns  to  cope  with  its  problems  of  external  adaptation  and 
internal  integration"  (p.  173).  Simply  stated,  culture  is  the  behaviors  and  beliefs  that  exist  in  an  organization.  Any 
change  is  an  assault  on  those  beliefs.  Some  will  "buy-in"  to  the  change  quickly  while  others  will  be  "middle 
adopters"  or  "laggards"  (Dormant,  1992,  p.  1 81 ). 

As  part  of  the  cultural  change,  show  that  the  technology  plan  is  important  to  the  district.  Organize  a 
parade;  set  off  metaphorical  fireworks  or  real  ones,  if  affordable;  have  a rally;  or  merely  have  the  band  play.  Show 
everybody  in  the  school  community  the  importance  of  the  technology  plan.  Sell  the  sizzle,  as  Burrus  (1993)  stated, 
and  be  prepared  to  deliver  the  thick,  juicy  steak  of  student  learning  and  goal  achievement. 

Communication  is  important  in  this  cultural  change.  The  committee  should  institute  a means  of 
communication  with  the  constituent  community.  Committee  members  who  serve  as  technology  hubs  is  one  avenue 
of  communications.  They  should  request  time  in  faculty  meeting  to  explain  planning  updates  to  teachers.  Staff  and 
community  newsletters,  brochures,  and  press  releases  are  other  means  of  communications.  Planning  committees 
should  exploit  these  means  of  informing  the  greater  school  community  of  progress  in  planning. 

Another  human  element  is  staff  development  or  "human  infrastructure."  What  good  is  a fully-funded 
technology  initiative  with  all  the  physical  hardware  and  wire  infrastructure  in  place,  if  the  staff  and  sludents  have  not 
been  trained  in  how  to  use  for  its  desired  purpose?  Burrus  states  (1993),  "Upgrade  technology  and  upgrade  people. 
The  two  must  go  together.  ...  If  [people]  don’t  have  the  necessary  skills  then  even  the  most  sophisticated  technology 
is  only  a blunt  instrument"  (p.  107). 

Burrus  (1993)  relates  a looming  problem  that  accompanies  the  lack  of  employee  training. 
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Talk  about  death  spirals.  Add  new  technology  at  great  expense  but  without  upgrading  the  people  who  will 
use  it.  When  the  technology  doesn't  work  right,  dispose  of  the  people.  The  problem  persists,  eliminate 
more  people.  No  solution  in  sight,  scrap  the  new  technology,  (pp.  107-108) 

While  education  depends  heavily  on  people  to  deliver  its  services,  the  business  problem  Burrus  explains 
has  a parallel  in  education;  tha’  is,  when  employees  and  students  arc  not  trained  and  consequently  do  not  use  the 
equipment,  blamed  is  assigned,  and  funding  is  diverted  to  other  projects. 

One  component  of  staff  development  is  related  to  the  organization's  reward  system.  Burrus  (1993)  listed 
many  short,  seemingly  trite,  sayings  about  causing  change,  but  one  is  important  for  this  circumstance.  "We  get  the 
behavior  we  reward"  (p.  179).  Burrus  maintains  that  the  quiet  ;st  way  to  get  a change  in  culture  is  to  change  the 
reward  system.  Reward  those  employees  who  first  adopt  what  the  new  culture  of  technology  offers.  As  other 
employees  see  that  the  new  way  is  rewarded,  they  will  begin  to  work  toward  those  goals,  too,  or  they  may  leave  the 
organization,  if  the  goals  are  contrary  to  their  personal  beliefs. 

Gilbert  (1978)  described  the  importance  of  reward  systems  many  years  ago.  He  divided  them  into  two 
areas:  "money  and  other  things"  (p.  309).  Other  things  are  "recognition  for  good  work,  patting  them  on  the  back, 
and  presenting  medals,"  among  many  possibilities.  Gilbert  advocated  using  money  to  reward. 

In  educational  organizations  the  reward  system  is  typically  in  the  form  of  recognition.  It  could  be  in  the 
form  of  a certificate  presented  in  a large  group  meeting;  a verbal  commendation  in  front  of  a large  group;  or  a letter 
of  commendation.  The  achieved  goals  should  become  part  of  individual  employee  evaluations,  although  this  is  a 
personnel  issue  and  rarely  seen  by  anyone  besides  the  teacher  and  the  evaluator. 

The  final  element  in  the  web  is  research  and  development  (R&D).  The  researcher  considers  the  R&D 
element  of  technology  planning  vital  in  part  of  the  planning  process.  With  the  high  costs  of  hardware,  software  and 
time,  (c.g.,  U.S.  Congress,  i 988),  planning  should  have  a speed  regulator  ~ the  R&D  section.  R&D  should  be 
directed  by  the  district's  technology  specialist  and  funded  as  part  of  the  technology  budget. 

The  R&D  component  has  at  least  two  branches.  One  of  them  deals  directly  with  attainment  of  goals,  and 
the  second  tests  new  products.  If  the  planning  process  centers  on  literacy  issues,  why  change  from  current  practices, 
if  the  learning  gains  from  educational  technology  are  minimal  and  there  is  no  other  measurable  advantage?  For 
example,  does  the  software  program  "Mathblaster"  contribute  to  student  growth  in  math  facts  belter  than  traditional 
means  of  drill  and  practice  activities  - such  as  flash  cards  and  chalkboard  contests?  If  R&D  finds  that  90%  of  third 
graders  know  90%  of  their  required  math  facts  after  using  "Mathblaster"  three  times  a week  for  20  minutes  each 
session  and  the  students  arc  achieving  at  the  75%  level  through  traditional  means,  this  finding  needs  to  be  explained 
to  teachers  through  many  sources,  especially  staff  development  activities  and  internal  ccmmunicalions.  Advantages 
of  using  this  software  need  to  be  identified  and  expressed  to  the  teachers.  Furthermore,  the  reward  and  employee 
evaluation  systems  needs  to  encourage  "Mathblastcr's"  use.  Additionally  R&D  could  assist  curriculum  adoption 
committees  as  they  test,  develop,  and  evaluate  prototypes;  and  assist  in  curriculum  development  activities. 

In  the  second  branch,  new  hardware  and  software  arc  tested.  As  new  hardware  is  introduced  to  the  market, 
the  R&D  section  should  acquire  it  and  develop  tests  to  sec  if  it  improves  achievement  in  stated  goals,  such  as  with 
the  previous  "Mathblaster"  example,  and  to  find  new  equipment's  operating  envelope.  This  important  step  identifies 
the  unit's  actual  operation,  not  what  the  sales  person  or  specifications  predict.  Buy  one,  put  it  in  a common  area  for 
students  and  staff  to  use,  and  let  them  help  determine  its  actual  operating  characteristics.  This  could  save 
embarrassment  to  tire  district  when  committing  money  to  buying  many  pieces  or  a system  for  the  district,  only  later 
to  find  out  it  does  not  operate  like  it  did  in  the  demons’  tions  or  as  the  specifications  promised. 

Educational  technology  planning,  as  any  oilier  planning  process,  is  not  a simple,  linear  function.  The 
Technology  Planning  Web  is  a research-based  template,  founded  on  the  insights  developed  during  this  study.  By 
using  tin  web,  planners  can  focus  on  goals  and  support  those  goals  with  full  consideration  of  staff  development, 
hardware,  finances,  R&D,  physical  infrastructure,  political  and  cultural  infrastructure,  and  evaluation. 
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Abstract 

This  study  examined  the  effects  of  an  inductive  multimedia  program,  including  graphs,  on  subjects ' ability 
to  create  linear  functions  and  conceptualize  variables  from  word  problems.  The  subjects  were  98  undergraduate 
students,  enrolled  in  two  sections  of  a computer  literacy  course  at  a large  southwestern  university.  Students' 
achievements  were  assessed  via  12-item,  short  answer , pre - and  posttests,  which  were  parallel  to  the  instructional 
themes  stated  in  the  treatment  programs. 

Students  were  randomly  assigned  to  one  of  two  treatment  groups  to  view  a version  of  a self-paced  program. 
Treatment  1,  the  inductive  table-only  program,  included  tables  in  addition  to  other  multimedia  elements  and  was 
developed  based  on  many  instructional  strategies  including  inquiry  inductive  learning  strategy.  Treatment  2,  the 
inductive  table-and-graph  program,  included  both  tables  and  graphs  and  was  identical  to  the  first  version  in  all 
respects  except  for  the  addition  of  the  graph  visuals  and  graph-related  text. 

Students,  regardless  of  treatment,  scored  significantly  higher  on  posttest  than  pretest  on  both  function 
construction  and  variable  conceptualization.  These  results  may  have  been  influenced  by  instructional  strategies 
used  in  the  treatments  including:  inquiry  mathematical  thinking,  schema  training,  linked  representational  systems, 
and  the  coordinate  graph  tutorial  teaching. 

Students  receiving  instructions  via  the  inductive  table-and-graph  program  scored  significantly  higher  on 
function  construction  section  of  the  posttest  than  did  students  receiving  the  table-only  treatment.  This  result  is 
consistent  with  prepositions  recognizing  the  conceptual  richness  of  visuals,  specifically  the  coordinate  graph,  in 
mathematics  education  learning. 

Results  from  the  present  study  suggest  the  use  of  inductive  multimedia  programs  treatments  that  include 
many  strategies  including  inquiry ■ learning  from  data,  tutorial,  schema,  and  core  representational  systems  for  the 
problem  of  translation,  specifically  creation  of  linear  function.  The  data  specifically  suggest  that  the  inductive 
multimedia  programs  that  include  the  coordinate  graph  tutorial  strategy  in  their  construction  have  a significant 
effect  on  the  function  construction  tasks. 

Introduction 

National  reports  (Dosscy,  Mullis,  Lindquist,  & Chambers,  1988;  Mullis,  1994;  Mull  is,  Dosscy,  Owen,  & 
Philips  1993)  indicate  that  many  students  do  not  understand  mathematical  concepts  and  skills  taught  in  schools. 
Mathematics  proficiency  is  crucial  for  the  individual  pursuing  higher  education  (National  Research  Council,  1989) 
and  critical  to  the  creation  of  an  informed  citizenry  and  economically  competitive  society  (Anderson  et  alM  1994). 

This  lack  of  mathematics  understanding,  in  algebra  for  example,  forces  students  to  memorize  algebraic 
rules  and  procedures.  Therefore,  many  students  think  algebra  is  simply  rule-based  memorization  (Brown  ct  al.  1988; 
Kicran,  1992).  As  a result,  students  are  often  unable  to  apply  basic  algebraic  and  geometric  concepts  to  problem 
solving  (Brown  et  al.).  A number  of  studies  indicate  that,  even  in  ordinary  relational  word  problems,  students  have 
major  difficulty  generating  equations  that  represent  relationships  (English  & Warren,  1995;  Chaiklin,  1989; 
Clement,  1982;  Herscovics,  1989;  Lewis  & Mayer,  1987;  MacGregor  & Stacy,  1993;  Mayer,  1982). 

Researchers  have  pointed  out  that  instruction  in  generating  linear  function  needs  an  approach  that  considers 
students'  general  reasoning  processes  (English  & Warren,  1994)  and  accounts  for  conceptual  errors  in  problem 
transition  (Kaput  & Simes-Knight,  1983).  In  addition,  instruction  must  include  linked  multimedia  presentation, 
emphasizing  visual  algebraic  process  representations,  including  tables,  patterns,  and  graphs  (Kaput,  1992;  Kosslyn. 
1980).  To  address  the  difficulty  students  have  in  learning  how  to  generate  linear  functions  (or  equations)  from 
relational  word  problems,  the  following  offers  a two-part  solution.  Using  linked  representational  media  (multimedia 
programs),  mathematics  Computer-Based  Instruction  (CBI)  should  stress  (1)  inductive  problem-solving  strategies  or 
scientific  heuristics  (for  example,  by  working  backwards,  working  inductively,  or  applying  algebraic  thinking  to 
data)  (DeMarois,  McGowen,  & Whitkanack,  1996;  Polya,  1954)  and,  most  importantly,  (2)  the  teaching  of  the 
language  of  mathematics  and  math  visuals  (graphs)  (Esty,  1992;  Kaput,  1992;  Bell  & Janvier,  1981).  Based  on  an 
inductive  multimedia  program  developed  according  to  the  above  approach,  this  study  sought  to  determine  how 
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visuals  (table  and  graph)  affect  students’  creation  of  linear  function  construction  and  variable  conceptualization  from 
relational  word  problems. 

Justification 

In  today's  work  environment,  the  ability  to  think  critically,  to  communicate  mathematical  ideas,  and  to 
develop  problem-solving  strategies  is  essential  (Smith,  1994).  Furthermore,  while  building  toward  a career,  research 
indicates  that  a strong  relationship  exists  between  mathematical  skills  and  success  in  college,  regardless  of  major 
(Waits  & Demana,  1988).  However,  despite  the  proven  short-  and  long-term  value  of  math  skills,  student 
underpreparedness  in  mathematics  is  a continuing  and  growing  problem  in  higher  education  (Berenson,  Best,  Stiff, 
& Wasik,  1990).  In  algebraic  problem-solving  situations,  students  find  algebraic  applications  difficult.  Collegiate 
mathematics  education  research  (Clement,  1982;  Lewis  & Mayer,  1987;  Lochhead  & Mestre,  1988;  Mayer,  1982; 
Wollman,  1983)  indicates  that  most  students  cannot,  in  fact,  translate  rational  word  problems  into  simple  linear 
functions. 

Several  researchers  (Bishop,  1989;  Clements,  1982;  Janvier;  1987)  have  encouraged  research  in  the 
exploration,  investigation,  and  curriculum  implications  of  graphical  and  tabular  representations  of  knowledge. 
According  to  Dugdale,  Thompson,  Harvey,  Demana,  Waits,  Kicran,  McConnell,  and  Christmas  (1995),  a graphical 
representation  "can  reveal  insights  into  the  problem  situation  that  are  not  readily  revealed  by  symbol  manipulation 
alone"  (p.  330).  On  the  other  hand,  studies  (Carpenter,  Corbit,  Kepner,  Lindquist,  & Reys,  1981 ; Goldenberg,  Lewis, 
& O'Keefe,  1992;  Kerslake,  1981;  Monk,  1992)  indicate  that  graphical  presentation  adds  its  own  ambiguity  to  the 
learner's  syntactic  translation  problem.  Despite  the  ambiguous  role  of  graphical  presentations  in  problem  solving, 
various  researchers  support  the  use  of  visuals:  as  mnemonic  tools  (Atkinson,  1975;  Lewis,  1989;  Winn,  Tian-Zhu,  & 
Schill,  1991),  as  gestalt-producing  mental  processors  (Skemp,  1989),  and  as  mathematics  language  (Esty,  1992; 
Janvier,  1987;  Kaput,  1989).  Graphs,  for  example,  enable  problem  solving  because  a graph  allows  one  to  view  a 
single  graphical  entity  instead  of  a binary  quantitative  relationship  (Kaput,  1989),  which  is,  by  definition,  a more 
complex  matrix  from  which  to  draw  a solution. 

Kosslyn  (1994)  has  shown  that  human  visual  perception  and  cognition  have  strengths  and  limitations,  and 
their  measures  depend  on  both  the  quality  of  visuals  displayed  and  the  adequate  usage  of  those  visuals.  With  their 
visual  (graphic)  and  authoring  programs,  low-cost  microcomputers  have  removed  many  visual  cognitive  obstacles  in 
mathematics  teaching,  as  in  their  numerous  other  applications  developed  to  aid  cognition.  Computers  may  help  us  to 
use  visuals  more  adequately.  For  example,  graphs  can  be  made  even  more  cognitively  engaging  via  animated 
displays  and  linked  explanations.  Authoring  programs  facilitate  development  of  materials  that  allow  visual  learning 
as  well  as  inductive  reasoning.  Computers  can  be  used  to  develop  visuals  and  support  reasoning  processes,  as  well 
as  to  measure  whether,  based  on  a set  of  dependent  variables,  visual  treatments  differ  on  average. 

Although  researchers  have  not  taken  up  his  mandate,  Clements  (1932),  a recognized  expert  in  mathematics 
education,  has  concluded  that  despite  the  fact  that  clear  guidelines  for  the  use  of  visuals  in  classroom  practice  have 
not  yet  emerged,  "there  should  not  be  a reduction  in  the  amount  of  research  which  is  aimed  at  achieving  this  end"  (p. 
36).  Clements  stresses  that  ‘’Mathematics  educators  need  to  develop  better  instruments  for  assessing  the  role  of 
visual  imagery  in  mathematics  learning”  (p.  36).  The  classroom  application  of  computer-supported  or  computer- 
generated  graphs  and  of  graphic  language  are  important  and  viable  areas  of  research  in  mathematics  teaching  (Bell 
& Janvier,  1981;  Bishop,  1989;  Drefus  & Eisenberg,  1987;  Eiscnberg  & Drefus,  1991;  Lcsh,  1987).  Currently, 
however,  little  research  exists  on  (1)  combined  visual  effects,  (2)  inductive  software  program  training.  (3)  strengths 
and  limitations  regarding  the  order  of  visual  presentation,  or  (4)  any  combination  of  the  above.  Since  the  use  of 
visual  thinking  in  mathematics  learning  is  controversial,  multivariate  research  considering  many  factors  may  help 
clarify  the  role  or  roles  of  visuals  in  problem  solving. 

The  present  study  considers  a multivariate  approach  to  the  role  of  visuals  in  problem  solving  based  on  the 
need  that  exists  for  inductive  algebraic  multimedia  software  programs  that  strongly  support  the  growing  interest  in 
the  use  of  visuals  in  mathematics-based  cognitive  processes.  The  graphical  presentation  and  construction  capabilities 
of  software  programming  currently  offer  the  most  practical  way  of  producing  and  using  quality  visuals.  The  mere 
inclusion  of  graphs  is  not  enough,  however;  programs  should  also  address  the  language  of  graphs  and  provide 
solutions  to  students1  graphical  misconceptions  in  a dialogue  with  learners.  Algehraic  software  programs  should 
consider  graphical  cognitive  obstacles,  graph  language,  and  the  order  of  visual  presentations.  The  present  study 
looks  for  a strong  possible  visual  effect  via  the  combination  of  tables  and  graphs  with  graph  language  in  an  inductive 
multimedia  program  to  improve  linear  function  construction  and  conceptualization  of  variables. 
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Hypotheses 

Hypothesis  1:  Students  receiving  instructions  via  cither  software  program  will  score  higher  on  the  posttest 
than  on  the  pretest  in  both  areas:  linear  function  and  vamblc  conceptualization. 

Hypothesis  2:  Students  receiving  instructions  via  the  inductive  table-and-graph  program  will  scoic  higher 
on  the  posttest  in  both  areas  than  will  students  receiving  the  table-only  treatment. 

Subjects 

The  subjects  were  98  undergraduate  students  enrolled  in  one  of  two  sections  of  EMC  321,  Computer 
Literacy,  offered  by  the  College  of  Education  during  the  Fall  1997  semester.  Subjects  ranged  in  age  from  18  to  25 
years. 

Treatments 

Two  inductive  multimedia  programs  served  as  the  instructional  treatments  for  this  study.  Both  self-paced 
treatments  were  developed  by  the  author  using  Authorware  3.0  via  Macintosh.  Both  programs,  InductiveThinker 
Table  and  InductiveThinker  Table  & Graph , had  two  lessons.  Lesson  one  contained  information  about  the  input, 
output,  and  independent  and  dependent  variables.  Lesson  two  included  information  about  the  rate  of  change  (or  the 
slope  of  the  function)  and  linear  function  creation.  Some  screens  were  added  to  InductiveThinker  Table  to  construct 
InductiveThinker  Table  & Graph.  The  pace  was  user  controlled  and  subjects  had  the  ability  to  navigate  between 
pages,  sections,  or  lessons  and  could  exit  the  program  at  any  time. 

The  instructional  treatments  were  evaluated  via  two  types  of  formative  evaluation  procedures  described  by 
Dick  and  Carey  (1985):  one-to-one  and  small-group.  After  a review  by  three  College  of  Education  instructors,  which 
suggested  a major  change  regarding  method,  level  of  control,  and  content,  the  treatments  were  piloted  using  seven 
students  similar  to  the  target  population  who  were  unfamiliar  with  the  programs. 

Instruments 

Two  instruments  were  used  to  collect  data: 

Instrument  I:  As  one  of  the  primary  measures,  the  first  instrument  was  a 12-item  pretest  administered  to 
both  groups  at  the  beginning  of  the  data  collection.  Test  items  were  in  short-answer  form  (both  English  and 
algebraic)  and  were  parallel  to  the  practice  items  contained  in  InductiveThinker  Table , the  first  program.  The  pretest 
items  measured  student  achievement  on  the  instructional  themes  specifically  stated  in  the  InductiveThinker  Table 
program. 

Instrument  2:  Another  of  the  primary  measures,  the  second  instrument  was  a 12-item  posttest  administered 
to  both  groups  at  the  end  of  the  treatment.  Test  items  were  again  in  short-answer  form  (both  English  and  algebraic) 
and  were  parallel  to  both  the  items  contained  in  the  pretest  and  to  the  practice  items  contained  in  InductiveThinker 
Table , the  first  program.  The  first  six  questions  of  both  the  pretest  and  the  posttest  paralleled  the  classic  student- 
professor  example,  to  some  extent.  The  last  six  items  of  both  the  pretest  and  the  posttest  were  related  to  the 
conceptualization  of  variables. 

Procedures 

Subjects  were  randomly  assigned  to  two  treatment  groups.  Both  treatment  programs  were  delivered  via 
Macintosh  computers  in  Farmer  2M,  a computer  lab  in  the  College  of  Education.  Throughout  the  self-administration 
of  the  program,  subjects  spent  as  much  time  as  they  desired  on  any  portion  of  the  program.  Subjects  from  each 
group  viewed  the  treatments  individually,  under  the  same  physical  conditions,  and  all  received  extra  credit  for 
participating  in  the  study.  Subjects  chose  their  own  participation  times  from  a variety  of  times  offered  and  were 
evaluated  via  a pretest  before  the  treatment  and  a posttest  upon  completion  of  the  programs. 

Design 

This  study  used  a Pretest-Posttest  Control  Group  design.  A pretest  of  the  dependent  variable  was 
administered  to  both  groups.  Then,  the  control  group  received  an  inductive  table  treatment,  while  the  experimental 
group  received  a manipulated  treatment  (Inductive  Table  and  Graph).  Finally,  both  groups  were  posttested.  The 
treatment  forms  were  A versus  A+B. 
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Validity  of  the  Research  Dr  ign 

The  Pretest- Posttest  Control  Group  design  was  selected  because  the  subjects  had  a variety  of  mathematical 
backgrounds  and  came  from  different  disciplines.  The  combination  of  (1)  random  assignment,  (2)  the  presence  of  a 
pretest,  (3)  the  presence  of  a control  group,  and  (4)  the  short  data  collection  period  controls  for  threats  to  internal 
validity,  such  as  statistical  regression,  mortality,  maturation,  history,  testing,  and  instrumentation. 

Statistical  Analysis 

Multiple  Analysis  of  Covariance  (MANCOVA)  was  used  to  determine  whether  there  existed  a significant 
difference  between  the  posttest  scores  of  subjects  who  received  Inductive  Table  instruction  and  those  vho  received 
Inductive  Table  and  Graph  instruction. 

Limitations 

The  sample  for  this  study  was  selected  from  two  sections  of  an  undergraduate  course  offered  by  the  College 
of  Education  at  Arizona  State  University.  The  three-credit  course  was  offered  during  the  Fall  1997  semester. 
Selection  was  based  on  the  willingness  of  instructors  to  facilitate  the  study  and  students  to  participate  in  the  study.  In 
addition,  only  undergraduates  enrolled  for  credit  (rather  than  audit)  w^re  selected.  Subjects  were  randomly  assigned 
to  one  of  two  groups. 

Significance  of  the  Study 

Significantly  higher  test  scores  from  those  students  who  received  the  Inductive  Table  and  Graph  instruction 
(1)  would  lead  to  a useful  guideline  for  computer-based  classroom  and  at-a-distance  algebraic  practices  and  (2) 
would  provide  positive  support  that,  when  they  are  taught  and  used  properly,  graphical  presentations  offer  a 
powerful  holistic  visual  aid  to  the  complex  abstraction  of  mathematics.  In  addition,  this  study  may  introduce  support 
for  the  role  of  graphs  in  theories  of  imagery. 

Results 

The  purpose  of  this  study  was  to  determine  the  effects  of  an  inductive  multimedia  program  including 
graphs  and  tables  on  subjects'  ability  to  create  linear  functions  and  construct  variables  from  word  problems.  Subjects 
viewed  one  of  two  self-paced  inductive  multimedia  programs.  The  first  program,  InductiveThinker  Table , included 
tables  in  addition  to  other  elements  (e.g.,  animation).  The  second  version  of  the  program,  InductiveThinker  Table  & 
Graph , included  both  tables  and  graphs.  This  program  was  identical  to  the  first  version  in  all  respects  except  for  the 
addition  of  the  graph  visuals  and  graph-related  text. 

Demographic  Data 

The  data  collection  began  with  101  undergraduate  students  enrolled  in  two  sections  of  Computer  Literacy,  in  a large 
South  western  university.  The  scores  of  three  students  were  discarded  because  they  did  not  finish  the  posttest.  uf  the 
remaining  98  students,  69  were  females  and  29  were  males  (70%  and  30%  respectively)  with  mean  age  of  22.6  years 
for  both  males  and  females.  Thirty-six  percent  of  students  reported  majors  in  Communication  or  Journalism,  23%  in 
education,  1 0%  in  justice  studies,  and  3 1 % others. 

Subjects  were  randomly  assigned  to  the  treatments  and  the  size  of  the  sample  was  not  fixed,  resulting  in 
groups  of  different  sizes.  The  size  of  the  sample  for  treatment  1,  Inductive  Thinker  Table , '^as  47  subjects  and  for 
treatment  2,  Inductive  Thinker  Table  and  Graph , 51  subjects. 

A split-half  (odd-even)  reliability  test  was  conducted  on  the  12  items  pretest  and  posttest  to  assess  linear 
function  construction  and  variable  conceptualization.  Coefficient  alphas  were  computed  to  obtain  internal 
consistency  estimates  of  reliability  for  these  two  scales.  The  alphas  for  the  pretest  and  posttest  scales  were  .70  and 
.61  respectively.  The  tests  measured  the  intended  content  areas:  Function  construction  and  variable 
conceptualization.  Pretest  and  posttest  items  were  parallel  to  the  practice  items  contained  within  the  instructional 
programs  and  were  short-answer  and  multiple  choice  in  format. 

Subjects  were  schedule  to  spent  one  hour  on  treatments.  However,  the  lab  observations  indicated  that 
subjects  spent  less  than  one  hour  with  the  programs.  Subjects  often  came  late  and  rushed  through  to  finish  the 
programs  in  order  to  catch  their  next  class.  Most  students  came  during  the  lab  rush-hour,  often  during  times  when 
they  were  supposed  to  be  in  their  computer  literacy  class.  Many  subjects  were  bored  and  tired  at  the  end  of  the 
treatments. 
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Data  Analysis 

Two  hypotheses  were  considered,  The  first  involved  examining  whether  the  mean  difference  between 
pretest  and  posttest  scores  on  two  levels  (function  construction  and  variable  conceptualization)  were  significantly 
different  from  zero.  One-way,  repeated-measure,  analysis  of  variance  (ANOVA)  was  used  to  measure  mean 
differences  between  pretest  and  posttest  scores,  first,  of  function  construction  and  second,  of  variable 
conceptualization. 

The  second  hypothesis  involved  examining  whether  the  posttest  adjusted  means  on  a set  of  dependent 
variables  (function  creation  and  variable  conceptualization)  varied  significantly  across  the  two  factors  (Table  and 
Table  & Graph  treatments).  A multivariate  analysis  of  covariance  (MANCOVA)  was  used  to  determine  whether 
groups  differed  on  the  dependent  variables.  The  Statistical  Package  for  the  Social  Sciences  (SPSS)  was  selected  for 
use  in  this  study.  An  alpha  level  of  .05  was  used  for  all  statistical  tests. 

Hypothesis  1 

Students  receiving  instructions  via  either  software  program  will  score  higher  on  the  posttest  than  on  the 
pretest  in  both  areas:  linear  function  and  variable  conceptualization. 

A one-way  repeated-measure  ANOVA  was  conducted  with  the  factor  being  either  treatment  and  the 
dependent  variable  being  the  pretest  and  posttest  scores  for  function  construction.  The  means,  highest  possible 
scores,  number  of  subjects,  and  standard  deviations  of  scores  are  presented  in  Table  1 . The  results  for  the  ANOVA 
showed  a significant  treatment  effect  on  function  construction  regardless  of  kind  of  effect,  Wliks’s  Lambda  = .457, 
F(l,  97)  1 15.27,  p < .001,  multivariate  d (Eta  Squared)  = .543. 

Table  /.  Means,  Highest  Possible  Score,  Number  of  Subjects,  and  Standard  Deviations  for  Function  Construction 
Scores 


Dependent  Variable 

M 

Highest  Score 

SD 

Pretest 

1.97 

6.00 

98 

1.79 

Posttest 

4.11 

6.00 

98 

1.52 

A follow-up  pairwise  comparison,  univariate  test  was  conducted.  The  results  confirmed  the  Multivariate 
test  indicating  that  the  posttest  mean  (M  = 4.1 1,  SD  = 1.52)  was  significantly  greater  than  the  pretest  mean  (M  1.97, 
SD  = 1 .79),  t (97)  = 10.74,  p < 0.008. 

Another  one-way  repeated-measure  ANOVA  was  conducted  with  the  factor  being  treatment  and  the 
dependent  variable  this  time  being  the  pretest  and  posttest  scores  for  variable  conceptualization.  The  means,  highest 
possible  score,  number  of  subjects,  and  standard  deviations  of  scores  are  presented  in  Table  2.  The  results  for  the 
ANOVA  showed  a significant  treatment  effect  on  variable  conceptualization  regardless  of  kind  of  effect,  Wliks’s 
Lambda  = .622,  F (1,  97)  = 58.84,  p < .001,  multivariate  d (Eta  Squared)  = .378. 


Table  2.  Means,  Highest  Score,  Number  of  Subjects,  and  Standard  Deviations  for  Variable  Conceptualization 
Scores 


Dependent  Variable 

M 

Highest  Score 

N 

SD 

Pretest 

4.06 

6.00 

98 

1.21 

Posttest 

5.01 

-)(.0 

98 

0.83 

A follow-up  pairwise  comparison,  univariate  test  was  conducted.  The  results  confirmed  the  Multivariate 
test  indicating  that  the  posttest  mean  (M  = 5.0 1 , SD  = .83)  was  significantly  greater  than  the  pretest  mean  (M  = 4.06, 
SD=  1.21),  t (97)  = 7.68, pc. 001. 
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HypcthCvSis  2 

Students  receiving  instructions  via  the  inductive  tab! e -and- graph  program  will  score  higher  on  the  posttest 
in  both  areas  than  will  students  receiving  the  table-only  treatment. 

A MANCOVA  with  two  dependent  variables  and  two  covariates  was  conducted.  The  independent  variable, 
instructional  treatment,  included  two  levels:  inductive  table  and  inductive  table  and  graph  programs.  The  dependent 
variable,  poshest  scores,  also  included  two  levels:  function  construction  and  variable  conceptualization.  The 
covariates  were  pretest  scores  on  both  function  construction  and  variable  conceptualizations. 

The  MANCOVA  indicated  that  the  adjusted  population  mean  vectors  (posttest  scores)  were  significantly 
different  among  the  groups  at  the  .05  level  (F  = 4.77,  p < .001).  The  MANCOVA’s  first  assumption  (check  to  sec 
that  there  is  significance  relationship  between  the  dependent  variables  and  the  covariatcs)  was  verified.  Sample  size 
justified  using  two  covariates  (C  < 8.  where  C is  the  number  of  covariates).  Tables  3 and  4 show  the  means  and 
adjusted  means  for  treatments  of  the  set  of  dependent  variables  and  the  means  of  pretests. 


Table  3.  Means  of  posttests  and  pretests  for  Function  Construction 


Treatment 

Observed  M 

Adjusted  M. 

Inductive  Table  (Posltest) 

3.70 

3.69 

Inductive  Table  & Graph  (Posltest) 

4.50 

4.51 

Inductive  Table  (Pretest) 

1.99 



Inductive  Table  & Graph  (Pretest) 

1.97 

Table  4.  Means  of  posttests  and  pretests  for  Variable  Conceptualization 


Treatment 

Observed  M 

Adjusted  M 

Inductive  Table  (Posttest) 

4.83 

4.89 

Inductive  Table  & Graph  (Posttcst) 

5.12 

5.12 

Inductive  Table  (Pretest) 

4.03 

Inductive  Table  & Graph  (Pretest) 

4.67 

A separate  analysis  of  covariance  was  done  on  each  dependent  variable.  The  probability  indicated  that  only  the 
function  construction  variable  was  significant  at  the  level  .05  (Table  5),  The  power  on  the  variable 
conceptualization  was  .3 1 . Probability  of  students  receiving  instructions  via  cither  treatment  and  score  significantly 
different  on  the  posttest  in  variable  conceptualization  is  very  low.  Low  power  indicates  a low  probability  of  rejecting 
the  null  hypothesis. 


Table  5.  Analysis  of  Covariance  on  Dependent  Variables 


VARIABLE 

MS 

F 

n 

Power 

Function  Creation 

16.38 

8.3 

.81 

Variable  Conceptualization 

1.33 

2.2 

.139 

.31 

The  multivariate  test  for  the  homogeneity  of  the  regression  hyperplancs  was  not  significant  at  the  .05  level 
(F  = .237,  p_  < .917)  indicating  that  the  assumption  of  homogeneity  was  quite  tenable.  The  multivariate  F, 
corresponding  to  Wilk’s  lambda,  indicated  that  there  is  a significant  difference  between  the  set  of  dependent  and  the 
set  of  covariatcs  at  the  .05  level  (F  = 6.367,  p < .001).  Table  6 indicates  that  22.83%  of  the  within  variability  on 
variable  function  construction  is  accounted  for  by  two  covariatcs,  pretest  function  construction  and  variable 
conceptualization. 
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Table  6.  Univariate  Tests  for  Relationship  Between  Dependent  Variables  and  Covariates 


VARIABLE 

SS 

MS 

F 

P 

Function  Creation 

22.83 

11.42 

5.8 

.004 

Variable  Conceptualization 

8.68 

4.34 

7.3 

.001 

The  Bryant-Paulson  procedure  was  conducted  because  of  the  possibility  of  measurement  error  on  the 
covariant  of  low  or  questionable  reliability.  The  question  was  whether  there  was  a significant  difference  hciwcen 
the  adjusted  means  on  function  construction  for  the  groups.  Results  again  indicated  a significant  difference, 
Hotelling  Value  = .0001 , interpolated  critical  value  = 2.83,  BP  = 5.78. 

Discussion 

Hypothesis  1:  Overall  Treatment  Effects 

The  results  of  the  current  study  support  Hypothesis  I.  Students,  regardless  of  group  or  treatment,  scored 
significantly  higher  on  posttest  than  pretest  on  both  function  construction  and  variable  conceptualization. 

This  improvement  is  not  likely  due  to  learning  during  pretest.  The  pretest  did  not  measure  factual 
information  which  could  be  recalled.  Rather,  the  translation  problem  is  a cognitive  obsticle.  Subjects  needed  an 
effective  treatment  that  addressed  conceptual  understanding.  “Taking  a pretest  on  algebraic  equations  ...  is  much 
less  likely  to  improve  performance  on  a similar  posttest”  (Gay,  1992,  p.  304). 

The  results  of  this  study  arc  consistent  with  the  finding  of  Wollman’s  (1983)  study.  Using  a tutorial 
strategy,  Wollman  found  that  94  percent  of  his  subjects  constructed  the  correct  equation.  However,  later,  Wollman 
(1985)  agreed  that  his  subjects  could  have  used  the  checking  procedure  to  construct  equations  regardless  of 
conceptual  understanding.  Observations  conducted  during  the  current  study  revealed  that  students  did  not  check 
their  answers  during  the  tests  and  used  either  table  or  graphical  procedures  using  arbitrary  or  given  data  to  create 
their  equations. 

Results  also  indicated  that  students  did  significantly  better  on  the  variable  conceptualization  portion  of  the 
posttest.  Students’  success  with  variable  conceptualization  may  have  positively  impacted  their  success  with  function 
construction.  If  this  is  the  case,  then  Clement’s  (1982)  report  that  variable  understanding  is  one  of  the  key  issues  in 
successful  problem  solution  is  supported  by  the  current  study. 

The  results  might  have  been  influenced  by  the  combination  of  other  instructional  strategics  represented  by 
the  current  study’s  instructional  treatments.  These  instructional  strategics  include  learning  from  (I)  mathematical 
thinking,  (2)  schema  training  , (3)  iinked  representational  systems,  and  (4)  the  coordinate  graph. 

The  effects  of  mathematical  thinking  required  by  the  current  study’s  programs  might  have  caused  the 
improved  results  of  the  present  study.  Shocnfeld’s  (1992)  mathematical  thinking  is  embedded  in  engagement  in 
scientific  research,  the  science  of  patterns,  and  the  determination  of  regularities  in  systems.  The  methodological 
framework  of  the  InductiveThinker  treatments  included  these  attributes.  The  treatments  of  the  current  study  include 
the  inductive  methodology  of  constructivist  epistemology  for  reasoning  and  discovering  via  the  construction  of 
tables  of  variable  values. 

There  might  be  an  interaction  effect  between  the  improved  results  of  the  current  study  and  the  schema- 
training  nature  of  the  treatments.  The  current  study’s  treatments  include  schema  acquisition  that  eventually  provides 
rule  automation  and  strengthens  mctacognitional  skills.  By  using  tables  of  values  and  describing  steps  of  procedures, 
for  example,  treatments  increase  memory  demand  that  could  hinder  student’s  rearrangement  of  information  and 
ability  to  construct  equations. 

The  results  could  also  be  attributed  to  the  anchored  linkcd-rcprcscntational  systems  (Kaput,  1992)  property 
of  the  treatments.  The  current  study’s  treatments  use  an  inductive  tutorial  mode  to  propose,  measure,  and  evaluate 
working  hypotheses  by  representing  linked  representations  of  mathematical  notations,  including  table,  verbal,  and 
graph  representations.  The  programs  ask  learners  to  write  down  their  thinking  and  working  models,  then 
immediately  compare  what  they  have  written  with  appropriate  examples.  Students  receive  tabular,  graphical,  and 
numerical  feedback  instantly  during  mindful  engagement  with  the  treatment  programs.  In  addition  to  the  linkcd- 
represcntational  effect,  other  media  attributes  such  as  mind-machinc  collaboration  could  be  a positive  attribute. 


197 


0 


01 


Hypothesis  2:  Inductive  Table  versus  Inductive  Table  and  Graph 

The  significant  difference  is  not  likely  the  result  of  the  longer  treatment.  The  inductive  (able-and-grnph 
pregram.  Treatment  2,  included  the  coordinate  graph  strategy  in  addition  to  other  strategies.  By  teaching  the 
language  of  graph,  the  coordinate  graph  strategy  used  its  attributes,  including  geometrical  graphical  representation  to 
further  subjects’  conceptual  understanding  of  the  translational  task.  Students'  difficulty  with  translational  tasks 
results  from  their  use  of  natural  language  syntax  and  their  lack  of  conceptual  understanding  of  variables  and  function 
construction,  not  from  the  amount  of  instruction  received.  Persistence  of  translational  problems  has  been  detected 
among  freshmen  engineering  students  (Clement,  1982)  who  received  extended  mathematics  instructions.  The 
difference  is  likely  to  be  the  result  of  the  alternative  and  graphical  core  representation  of  the  coordinate  graph. 

The  multivariate  results  related  to  the  second  hypothesis  of  this  study  indicating  that  graph  has  a significant 
effect  in  function  construction  are  consistent  with  the  findings  of  Schwarz  ct  al.  (1990),  Tall  and  Thomas  (1989), 
and  Ycrushalmy  (1988). 

The  most  important  outcome  of  the  current  study  is  the  second  finding,  that  students  receiving  instructions 
via  the  inductive  lablc-and-graph  program  scored  significantly  higher  on  function  construction  of  the  poshest  than 
did  students  receiving  the  table-only  treatment.  Treatment  2,  InductiveThinker  Table  and  Graph , includes  all 
elements  of  the  first  treatment  as  well  as  graphical  representation  and  the  teaching  of  the  language  of  the  graph 
elements.  The  second  finding  is  attributed  to  the  graphic  strategy  of  Treatment  2 and  is  most  likely  related  to  the 
teaching  of  the  graphic  language  by  the  treatment  program.  Representing  linked  presentations,  including  graphics 
alone,  is  not  likely  to  be  a major  factor,  since  much  research  (Goldcnbcrg,  Lewis,  & O'Keefe,  1992;  Kerslakc,  1981 ; 
Monk,  1992)  indicates  that  graphic  mediation  has  its  own  ambiguity  that  adds  to  the  learner’s  syntactic  translation 
problem. 

Contrary  to  expectation,  investigation  of  the  second  hypothesis  revealed  that  the  groups  were  not 
significantly  different  regarding  variable  conceptualization.  One  possible  explanation  for  this  finding  is  the  length  of 
Treatment  2,  which  was  about  half  an  hour  longer,  and  the  fact  that  the  last  questions  of  the  posttest  were  related  to 
variable  conceptualization.  Observations  during  data  collection  revealed  that  most  students  rushed  through 
answering  the  posttest  questions;  many  came  late  and.  after  the  posttest,  had  to  leave  quickly  to  attend  classes  across 
campus.  Another  explanation  might  be  the  fa'’’  that,  according  to  a few  observation  notes,  posttest  and  pretest 
questions  were  very  similar  and  students  may  have  thought  they  were  practically  the  same  questions.  The  result 
could  also  be  attributed  to  the  possible  testing  and  pretest-treatment  interaction  effect.  A careful  reexamination  of 
the  measures  indicated  that  tables  that  were  used  in  pretests  for  explanation  of  the  variable  questions  were  not  used 
in  the  function  construction  section.  This  may  have  caused  students  to  score  higher  on  the  pretest. 

Another  possible  explanation  is  that  the  tests  might  not  have  been  extensive  enough  to  measure  variable 
conceptualization.  Figure  2 (in  Chapter  4)  indicates  a ceiling  effect  on  posttest  scores.  Majority  of  subjects  achieve 
the  highest  scores  possible. 

Implications  of  the  Study 

Results  from  the  present  study  suggest  that  for  the  problem  of  translation,  schools  may  find  it  most 
beneficial  to  use  treatments  similar  to  InductiveThinker  Table  and  Graph , that  employ  all  learning  strategies, 
including  inquiry  learning  from  data,  tutorial,  schema,  and  core  representational  systems.  The  present  study 
indicates  that  the  coordinate  graph  strategy  is  an  important  representational  system  and  is  very  effective  in 
translational  tasks  only  when  its  language  is  taught  by  software  programs  and  that  language  is  understood  by 
learners.  Otherwise,  the  implication  is  that  the  use  of  coordinate  graphs  simply  adds  to  the  learner’s  syntactic 
translational  problem.  Difficulties  regarding  graph  integration,  construction,  and  translation  must  be  recognized  and 
taught  to  learners  before  they  use  graphs. 

Before  students  learn  how  to  construct  functions  using  any  of  the  proposed  instructional  solutions  to  the 
problem  of  translation,  they  need  to  understand  the  concept  and  use  of  variables  (Usiskin,  1988).  This  study  did  not 
find  significant  differences  between  groups  in  the  area  of  variable  conceptualization;  subjects  earned  maximum 
scores  regardless  of  the  group  to  which  they  belonged.  However,  function  construction  includes  variable 
recognition.  Students  must  learn  to  conceptualize  variables  in  order  to  understand  functions. 

Strengths  and  Limitations 

This  study  considered  one  of  the  most  important  problems  in  the  teaching  and  learning  of  college  algebra: 
the  translation  task  in  word  problems.  This  study  indicated  that  translation  from  problem  situation  to  symbolic 
algebra  is  difficult  for  students,  and  that  linking  reality  and  mathematics  symbolism  requires  students  to  engage  in 
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various  stages  of  abstraction  that  include  interpretation,  construction,  and  translation.  This  study  combined 
instruction  treatment  types  examined  and  recommended  by  the  experts  and  investigated  their  combined  effects. 
Based  on  the  literature  review,  this  study  used  the  conceptual  richness  of  visuals  in  mathematics  education  — 
specifically,  the  coordinate  graph,  a powerful  representational  introduction  to  the  complex  abstractions  of 
mathematics. 

The  current  study  addresses  a solution  to  the  problem  of  student’s  translation  in  algebra  by  proposing  and 
evaluating  a solution  to  the  problem  of  function  construction.  This  study  introduced  a combined  instructional 
strategy  solution  that  includes  the  coordinate  graph.  The  combined  strategy  was  measured  against  another  combined 
strategy  without  the  coordinate  graph.  This  measurement  allowed  for  a more  meaningful  result  than  would  the 
comparison  of  two  treatments  reflecting  disparate  strategics.  The  current  study  used  a gestalt-producing  (Kaput, 
1989)  comb  ned  approach  by  including  the  coordinate  graph  in  the  study’s  proposed  multiple  strategies.  A graph 
allows  consolidation  of  pairs  of  related  numbers  into  points:  graphing  connects  algebra  to  geometry.  A strength  of 
the  study  was  that,  in  keeping  with  hypotheses  2,  all  the  effects  but  visual  effect  (the  coordinate  graph)  remained 
constant  in  order  to  measure  the  graphical  effect.  For  both  hypotheses,  the  study  used  univariate  as  well  as 
multivariate  analysis. 

Limitations  of  this  study  that  may  affect  validity  include  (1)  subjects’  voluntary  participation  (2)  the 
content  of  test  measures  regarding  the  conceptualization  of  variable,  and  (3)  limited  qualitative  observation  and 
analysis.  Although  the  test  items  regarding  the  conceptualization  of  variables  were  parallel  to  the  exercise  items  in 
the  programs,  more  challenging  conceptual  understanding  should  be  require  on  the  tests.  For  example,  instead  of 
providing  multiple-choice  questions  regarding  the  conceptual  understanding,  short-answer  responses  should  he 
designed.  More  through  qualitative  observation  and  analysis  would  indicate  the  particular  strategics*  students  used  in 
performing  translational  tasks. 

Recommendations  for  Future  Research 

Further  research  should  include  the  following: 

• Expanded  Designs:  Designs  that  include  a third  group  that  is  not  exposed  to  a treatment  to  reveal  possible 
pretest  interaction 

• Factorial  Designs:  Designs  which  consider  the  joint  effect  (joint  interactions)  of  the  treatments 

• Qualitative  Designs:  Designs  in  which  students’  mathematical  (translational)  thinking  is  observed,  and  in 
which  subjects  arc  interviewed  regarding  their  experience  with  the  treatments 

Conduct  experimental  research  that  considers  oilier  independent  factors  beside  treatments,  including  sex, 
IQ,  and  mathematical  background.  Such  research  should  measure  the  possible  joint  interactions  of  these  factors  on 
the  study’s  output.  The  strength  of  this  design  would  be  the  examination  of  a joint  effect  of  sex  and  treatment  on  the 
dependent  variahlc(s). Regarding  function  conceptualization,  Drefus  and  Eisenberg  (1983)  said,  “The  challenge  is 
clear;  the  problem  is  well  defined.  We  must  teach  so  that  our  students  will  be  able  to  grasp  global  notions  and  find 
inter-relationships.’’  Still,  a decade  later,  MacGregor  and  Stacey  (1993)  reported  that  “One  of  the  greatest  difficulties 
for  beginners  in  algebra  is  linking  a mathematical  situations  to  its  formal  description.”  Students  use  a naturalistic 
representational  system  — the  English  language  — to  translate  word  problems  to  functions  instead  of  using  a variety 
of  mathematics  representational  systems  that  includes  the  coordinate  graph.  As  shown  by  the  current  study,  using  a 
combined  treat  me/'  that  includes  the  coordinate  graph,  educators  can  help  students  to  overcome  the  difficulty  or 
misconception  of  the  ^slational  tasks. 
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Defining  an  environment 

As  computer  technology  increases  in  sophistication,  so  does  our  ability  to  provide  users  with  more 
sophisticated  learning  and  performance  environments.  Advances  in  the  conception  of  computer-based  environments 
have  included  Gery’s  (1991)  notion  of  electronic  performance  support  systems  (EPS S),  and  Hannafin’s  (1992) 
description  of  the  importance  of  computer-based  learning  environments  (CBLE).  CBLE’s  are  intended  to  be  a 
logical  progression  of  our  knowledge  of  learning  theory  and  the  capabilities  of  today’s  computer  systems. 

Creating  a learning  environment  is  not  easy.  Hannafin  (1992)  says  that  the  learning  environment  should  be 
a comprehensive  integrated  system  that  promotes  the  engagement  of  the  learner,  provides  for  the  manipulation  of 
information  contained  in  a variety  of  media  centered  around  a specific  learning  theme,  and  he  a rich,  exploratory, 
interactive  environment.  Rciber  ( 1996)  adds  the  important  element  of  the  learning  environment  being  self  regulated 
by  the  learner.  Finding  media  that  is  centered  around,  or  supports  a particular  learning  theme  is  not  difficult  to  do. 
Creating  rich,  exploratory  interactive  environments,  while  difficult,  is  not  impossible.  The  singularly  difficult  task  is 
building  an  environment  that  is  truly  engaging. 

What  is  engagement? 

Defining  engagement  is  difficult  because  of  its  tendency  to  be  relative  to  a particular  task.  There  are 
various  types  of  engagement.  In  a CBLE  we  arc  referring  to  the  notion  that  the  program  makes  the  learner  want  to 
be  there.  There  may  be  many  reason  why  the  learner  chooses  to  be  there.  The  ARCS  model  (Keller,  1983;  1988) 
offers  a detailed  discussion  of  the  varying  elements  in  the  learners’  motivation  to  learn.  One  reason  learners  choose 
to  work  in  a learning  environment  may  be  that  they  arc  simply  interested  in  the  content.  Interest  in  the  content,  or 
having  a well  defined  question,  provides  the  learner  with  intrinsic  motivation  to  he  working  within  the  environment. 
Learners  with  intrinsic  motivation  may  also  be  said  to  be  exhibiting  epistcmic  curiosity  (Gagne,  1985).  Epistcmic 
curiosity  is  often  caused  by  incongruous  ideas,  beliefs  or  attitudes  about  a subject,  which  is  to  say,  curiosity  based 
on  lack  of  knowledge  about  a particular  problem.  A learner  who  is  internally  motivated  may  find  that  something  as 
generic  as  a multi-media  encyclopedia  could  serve  as  a learning  environment.  The  motivation  comes  from  within 
the  learner,  and  the  environment  itself  does  not  necessarily  need  to  be  engaging  beyond  its  ability  to  present  the 
content  in  a factual  manner.  If,  however,  the  learner  is  not  intrinsically  motivated,  then  the  environment  may  need  to 
offer  greater  motivational  features  to  keep  the  learner  interested.  It  is  not  unrealistic  to  imagine  that  every  learner 
may  exhibit  intrinsic  motivation,  and  require  extrinsic  motivational  features  during  the  life  span  of  the  learning 
process,  or  while  they  arc  working  within  the  learning  environment.  Therefore,  I am  defining  engagement  here  as 
the  nexus  of  intrinsic  knowledge  and  or  interest  and  external  stimuli  that  promote  the  initial  interest  in,  and 
continued  use  of  a computer-based  learning  environment. 

Flow  Theory 

Complete  and  total  involvement  in  a given  task  is  defined  and  described  by  Csikszentmihalyi ’s  Flow 
Theory  (Csikszentmihalyi,  1990).  Flow  theory  is  described  as  the  feeling  of  optimal  experience.  It  is  felt  when 
“...instead  of  being  buffeted  by  anonymous  forces',  we  do  feel  in  control  of  our  own  fate.  ...we  feel  a sense  of 
exhilaration,  a deep  sense  of  enjoyment ...  (Csikszentmihalyi,  1990,  page  3).  In  order  to  reach  this  state  of  optimal 
experience:  “There  must  he  a goal  in  a symbolic  domain;  there  have  to  be  rules,  a goal,  and  a way  of  obtaining 
feedback.  One  must  be  able  to  concentrate  and  interact  with  the  opportunities  at  a level  commensurate  with  one's 
skills”  (Csikszentmihalyi,  1990,  page  118).  Flow  is  often  experienced  during  physical  activities  because  of  the 
ability  to  realize  the  manifestations  of  the  requirements  of  a flow  experience.  For  many  people  finding  a moment  of 
flow  can  he  when  they  arc  doing  things  that  they  enjoy  and  excel  at,  although  flow  can  be  experienced  by  nearly 
anyone  when  they  arc  competing  in  an  environment  that  is  appropriate  to  their  skills.  Basketball  players,  even  poor 
ones  like  myself,  report  games  when  everything  you  throw  up  goes  in.  It  is  when  it  is  your  day,  when  you  arc  so 
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totally  engrossed  in  your  activity  that  you  lose  yourself  for  the  duration  of  the  activity.  You  arc  in  the  zone . Though 
I might  experience  that  zone  on  the  playground  with  people  at  my  own  skill  level,  I am  not  likely  to  experience  flow 
in  a pick  up  game  with  NBA  All  Stars.  The  reason  for  this  is  that  despite  the  fact  that  my  level  of  play  might  rise  to 
the  occasion,  it  would  never  rise  far  enough  for  me  to  feel  comfortable.  The  zone  can  be  explained  through  flow 
theory.  While  flow  is  often  experienced  during  physical  activities,  it  occurs  for  different  people  during  different 
activities,  and  has  been  documented  to  occur  during  the  learning  process.  Most  often  flow  in  learning  occurs  at 
times  when  outside  forces  do  not  dictate  what  is  to  be  learned  (Csikszentmihalyi,  1990).  When  people  arc 
intrinsically  motivated,  they  find  themselves  able  to  “read  for  hours”  or  “pull  an  all  nighter”  to  master  the  content. 
But  what  happens  during  environments  where  there  is  no  intrinsic  motivation?  Is  it  possible  to  provide  learners  in  a 
contrived  environment  with  the  necessary  tools  to  reach  a state  of  flow  in  order  to  optimize  the  learning  experience? 

Video  Arcades  and  Learning 

As  an  undergraduate  in  1983  I watched  my  college  friends  get  in  the  zone  with  computer  games.  People 
standing  in  front  of  a video  game  console  would  flex  their  hands,  stretch  their  shoulders  to  loosen  up  to  play  a video 
game.  Once  in  the  game,  they  could  be  in  a Zen-like  state  of  complete  attachment  to  the  task  at  hand.  Never  mind 
that  the  task  was  silly.  Never  mind  that  they  themselves  would  likely  never  pilot  a space  ship  and  face  legions  of 
attacking  aliens.  Never  mind  the  fact  that  all  of  the  rules  were  arbitrary  and  contrived  by  someone  else — they 
played  these  games.  They  learned  every  subtle  nuance  of  the  games.  They  often  knew  more  about  the  games  than 
they  ever  knew  about  the  classes  they  were  taking.  Despite  the  fact  that  there  was  no  authentic  problem  for  them  to 
solve,  they  found  themselves  completely  engrossed  in  the  task. 

During  studies  of  games  (Jones,  1997)  I reported  that  certain  games  engrossed  people  so  totally  that  they 
could  not  stop  playing.  Herz  (1997)  describes  well  the  intense  fascination  people  find  in  computer  games.  It  is  not 
just  the  ability  to  play,  to  face  danger,  death,  and  ultimate  mayhem  and  still  come  out  alive.  It  is  not  solely  about 
competition.  It  is  about  the  intense  feelings  of  engagement  that  a “good”  game  can  instill  in  the  player.  In  previous 
and  ongoing  studies  of  computer  games  and  gamers,  I have  heard  stories  of  people  who  played  Mysl  for  an  entire 
weekend,  ignoring  the  need  to  sleep  in  order  to  solve  the  puzzles.  I have  spoken  with  people  who  can  tell  within  the 
first  few  minutes  of  starting  a game  whether  or  not  they  will  play  a particular  game  well.  “I  don’t  have  the  feeling 
today,”  w j the  comment  of  one  gamer.  There  exists  among  many  people  who  play  games  an  ability  to  become 
completely  in  touch  with  one’s  self  and  one’s  abilities.  This  is  due  in  part  to  one’s  ability,  and  in  part  due  to  the  fact 
that  there  exists  within  people  strong,  tangible  feelings  of  attachment  to  the  games  they  play.  These  games  engender 
deep  feelings.  Good  games  rock;  bad  games  suck.  One  of  the  key  reasons  that  a game  can  foster  this  kind  of 
devotion  is  that  good  games  tend  to  the  total  package.  It  is  not  just  the  graphics,  sounds,  and  other  multi  media 
assets  they  use.  It  is  about  how  those  assets  help  define,  support,  and  give  life  to  a domain  that  has  no  counterpart  in 
the  physical  world.  They  draw  you  in  and  make  you  believe.  Doom  and  Doom  H were  so  frightening  to  some 
players  that  they  refused  to  go  into  the  labyrinth  (i.e.  play  the  game)  again  (Herz,  1997).  That  is  a powerful 
statement.  These  arc  truly  immersive  environments  that  support  the  eight  major  components  of  the  How  experience. 
This  is  demonstrated  in  Table  1 . 

Enjoyment  and  Pleasure 

Within  Flow  Theory,  Csikzentmihalyi  (1990)  defines  the  difference  between  enjoyment  and  pleasure. 
Enjoyment  is  characterized  by  the  feeling  of  forward  movement,  or  a sense  of  accomplishment.  Pleasure  can  give 
enjoyment  and  can  even  contribute  to  enjoyment.  However,  pleasure  is  defined  as  being  passive  in  nature,  while 
enjoyment  requires  direct  participation  by  an  individual.  An  optimal  experience  is  more  of  a manifestation  of 
enjoyment  than  pleasure.  Being  active  in  the  experience  tends  to  promote  enjoyment.  It  is  analogous  to  the 
difference  of  watching  somebody  playing  basketball  and  playing  yourself.  The  former  is  nice  to  watch,  but  the  latter 
gives  you  a greater  feeling  of  accomplishment. 


Table  I.  Components  of  flow  as  manifested  in  computer  games. 


Element  of  Flow 

Manifestation  in  a game 

I . Task  that  we  can  complete 

The  use  of  levels  in  a game  provide  small  sections 
that  lead  to  the  completion  of  the  entire  task 

2.  Ability  to  concentrate  on  task 

Creation  of  convincing  worlds  that  draw  users  in. 
The  Dungeons  and  Labyrinths  in  Doom  II  help  you 
suspend  your  belief  systems  for  a time. 

3.  Task  has  clear  goals 

Survival,  collection  of  points,  gathering  of  objects 
and  artifacts,  solving  the  puzzle 

4.  Task  provides  immediate  feedback 

Shoot  people  and  they  die.  Find  a clue,  and  you  can 
put  it  in  your  bag. 

5.  Deep  but  effortless  involvement  (losing 
awareness  of  worry  and  frustration  of 
everyday) 

The  creation  of  environments  that  are  far  removed 
from  what  we  know  to  be  real  helps  suspend  belief 
systems  and  take  one  away  from  the  ordinary 

6.  Exercising  a sense  of  control  over  their  actions 

Mastering  the  controls  of  the  game,  such  as  mouse 
movement  or  keyboard  combinations 

7.  Concern  for  self  disappears  during  flow,  but 
sense  of  self  is  stronger  after  flow  activity 

Many  games  provide  for  an  environment  that  is  a 
simulation  of  life  and  death.  One  can  cheat  death 
and  not  really  die.  People  stay  up  all  night  to  play 
these  games.  It  is  the  creation  of  an  integration  of 
presentation,  problem,  and  control  over  the  system 
that  promotes  this. 

8.  Sense  of  duration  of  time  is  altered. 

Years  can  be  played  out  in  hours.  Battles  can  be 
conducted  in  minutes.  The  key  point  is  that  people 
can  stay  up  all  night  playing  these  games. 

In  computer  games,  certain  features  provide  pleasure:  good  graphics,  nice  music,  visual  effect.;,  and 
interesting  animations  arc  aesthetically  pleasing,  but  do  not  necessarily  contribute  to  creating  a good  game.  What 
makes  a game  “good”  is  a good  problem  that  is  manifested  appropriately.  It  does  not  matter  if  it  is  rendered 
elegantly,  but  that  it  is  rendered  in  a manner  consistent  with  the  problem.  Herz  (1997)  presents  convincing 
arguments  that  graphically  minimal  games  of  the  late  1970’s  and  early  1 980’s  (such  as  Tempest  and  Pac  Man)  were 
actually  more  engaging  than  more  recent  games  that  provide  a much  greater  level  of  graphic  detail.  Much  of  this 
has  to  do  with  the  problem  facing  the  user  and  how  it  is  manifested.  Problems  that  have  meaning,  that  stretch  one’s 
abilities  to  the  limits  arc  more  likely  to  provide  feelings  of  flow  than  easy  problems:  we  like  to  be  challenged.  If  we 
arc  to  relate  this  notion  to  the  development  of  learning  environments,  it  might  suggest  that  we  need  to  help  the 
learner  define  a problem  within  the  environment,  or  that  perhaps  the  environment  itself  could  represent  a 
manifestation  of  a problem.  This  might  be  accomplished  by  building  tools  into  the  learning  environment  that  help 
learners  solve  a problem.  Even  drill  and  practice  activities  can  help  maintain  a level  of  involvement  on  the  part  of 
the  learners. 

Cognition  and  Plow 

It  might  be  argued  that  one  could  make  a relationship  between  cognition  and  flow  as  it  relates  to  CBLE’s. 
Norman  (1993)  speaks  of  two  kinds  of  cognition:  experiential  and  reflective.  Experiential  cognition  is  one  where 
one  may  react  to  events  efficiently  and  effortlessly.  An  example  of  this  would  be  the  way  people  who  play  action 
games  such  as  Doom  II  exhibit  this  type  of  cognition.  It  is  a combination  of  skill,  reflexes,  and  knowledge  that 
comes  together  after  many  hours  of  practice.  Reflective  cognition  is  that  of  comparison  and  contrast  of  thought,  of 
decision  making.  It  is  the  type  of  cognition  that  leads  to  new  ideas  and  novel  responses.  Strategy  games,  such  as 
Warcraft  If  or  puzzle  games  such  as  Mysf  require  the  use  of  this  type  of  cognition.  In  working  with  complex  tasks, 
it  is  usually  necessary  to  combine  both  '-cflcctivc  and  experiential  cognition  to  solve  problems.  It  is  likely  that  in 
order  to  reach  flow  that  one  must  use  both  types  of  cognition.  One  would  need  efficient,  seemingly  effortless  skills 
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related  to  one’s  ability  level  (experiential  cognition)  and  the  ability  to  assimilate  and  accommodate  new  information 
(reflective  cognition)  in  order  to  do  most  activities  well  enough  to  reach  a state  of  flow. 

Relating  these  ideas  to  the  design  of  a CBLE  may  mean  that  lack  of  attention  to  aesthetics  may  make  a 
program  less  likely  to  be  used,  but  mere  inclusion  of  aesthetically  pleasing  elements  does  not  guarantee  that  the 
elements  will  promote  learning,- or  add  to  the  enjoyment  of  the  program.  Pleasure  can  be  had  or  developed  through 
seductive  bells  and  whistles  that  are  added  on  to  a program.  Enjoyment  might  occur  because  the  bells  and  whistles 
were  used  as  intentional  pieces  of  the  environment.  The  bells  and  whistles,  or  multi-media  assets,  employed  in  an 
environment  should  be  used  to  promote  the  workings  of  the  environment.  Music  that  is  used  in  games  can  help 
underscore  emotions.  The  fast,  hard  pounding  music  in  Doom  II  causes  one’s  adrenaline  to  increase  in  an  action 
game,  where  as  the  ephemeral  background  music  in  Myst  helps  enrich  the  feeling  of  mystery.  Beyond  the  ability  of 
the  multi-media  assets  to  support  enjoyment  by  helping  to  support  or  carry  forth  a consistent  tone,  enjoyment  can 
only  be  had  when  there  is  some  conflict  between  what  you  know,  and  what  you  want  to  know,  which  is  the  essential 
component  of  cognitive  conflict  (Piaget,  1 980;  Gagne,  1985).  Cognitive  conflict  stretches  our  desire  to  know  and  do 
more.  It  is  a confrontation  between  the  learners’  current  knowledge  and  the  learners’  expectations  or  ambitions.  It 
is  the  challenge  needed  to  begin  flow  experience.  A possible  way  to  build  this  conflict  into  a CBLE  is  through  the 
use  of  environmental  juxtaposition . 

Building  an  environment 

When  thinking  of  a learning  environment,  I prefer  to  work  with  Reiber’s  notion  of  an  endogenous  learning 
environment  (Reibcr,  1996).  In  these  environments,  the  content  and  its  structure  arc  so  closely  related  that  “one 
cannot  tel!  where  the  content  stops  and  the  game  begins”  (Reiber,  1996,  page  50).  One  place  where  this  notion  of 
endogenous  environments  is  most  notable  is  in  the  area  of  commercial  computer  games.  The  games  themselves  arc 
motivating,  and  weave  a fabric  of  content  and  fantasy  so  seamlessly  that  one  can  become  lost  in  the  game  for  hours. 

Environmental  Juxtaposition 

Much  of  the  content  of  a computer-based  learning  environment  is  presented  visually,  and  surrounded  by  or 
supported  by  some  type  of  theme.  Visually,  an  endogenous  environment  strives  for  seamless  integration  of  the 
program’s  theme,  content,  and  the  patterns  of  interaction  used  in  the  game  or  other  type  of  environment.  The 
original  pattern  of  interaction  in  a computer  game  was  that  of  slide  and  shoot.  Your  agent  in  the  game  resided  on  a 
horizontal  plain,  and  moved  left  to  right  while  attacking  agents  moved  on  a vertical  plain.  This  was  begun  in  Space 
Invaders , and  can  still  be  seen,  though  more  elegantly  rendered,  in  current  games  such  as  Mortal  Kombat.  This 
interaction  pattern  has  grown  into  a roam  and  shoot  pattern  of  interaction.  No  longer  must  you  wait  for  the  enemy  to 
come  to  you:  now  you  can  go  out  and  hunt  them  down.  Myst  defined  a new  pattern  of  interaction.  You  search  for 
clues  within  an  environment.  These  clues  help  not  only  to  solve  the  puzzle,  but  to  figure  out  how  the  world  works 
as  well.  Typically,  when  moving  within  a game,  the  game’s  controls  work  in  concert  with  the  game’s  content  to 
provide  a seamless  integration  of  content  and  control.  This  is  a good  thing.  However,  it  is  sometimes  desirable  to 
impose  some  type  of  juxtaposition  on  the  environment,  and  ultimately  the  learner. 

Occasional  juxtaposition  or  conflict  within  the  user  interface  can  keep  people  moving  and  engaged.  The 
Monty  Python  games  are  excellent  examples  of  this.  In  Monty  Python's  Quest  for  the  Holy  Grail , unexpected 
surprises  keep  the  users  engaged.  However,  one  might  argue  that  Monty  Python  products  would  not  be  complete 
without  significant  juxtaposition  between  the  environment  you  are  in  and  the  surprises  they  throw  at  you. 

Juxtaposition  integrated  between  content  and  control  might  suggest  that  stimulus  and  response  is  not  all 
that  bad  of  an  idea  in  a computer-based  learning  environment.  Early  computer-based  instructional  programs  were 
often  criticized  as  being  electronic  page  turners.  They  did  not  engage  the  learner  at  a level  beyond  passive  viewing. 
Learners  need  to  be  engaged  in  the  educational  process,  and  actively  engaged  in  the  content  and  the  business  of 
learning.  Learners  should  be  doing  things  in  the  software.  Clicking  on  timelines,  accessing  pop-up  text  or  graphics, 
clicking  and  dragging  objects  are  all  examples  of  active  techniques  used  in  CBLE's.  Additionally  building  tools 
within  the  learning  environment  extend  the  notion  of  educational  software  to  the  arena  of  application  software. 
Instead  of  being  simply  a reference  tool,  the  software  becomes  a tool  for  calculating,  comparing,  and  generally 
working  on  a problem.  Investigating  Lake  Huca  , a multi-media  learning  environment  created  at  the  University  of 
Wollongong,  Wollongong  , Australia,  was  me  of  the  first  pieces  of  software  I saw  that  did  this  (Sec  Figure  1). 

Tools  were  there  for  testing  soil  and  water,  notebooks  were  there  for  learners  to  write  down  notes  and  store 
information  to  study  later.  These  tools  take  away  from  the  visual  consistency  of  the  program.  In  a two  dimensional 
environment  that  is  displayed  on  a computer  screen,  it  is  necessary  to  treat  different  locations  through  the 
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functionality  of  multiple  windows.  The  tools  work  on  the  lake,  but  the  tool  pallctc  can  cover  other  areas  of  the 
screen.  However,  the  juxtaposition  they  provide  is  pcdagogically  significant,  relatively  seamless,  and  ultimately 
helpful  and  not  distracting  (See  Figure  1 ). 

In  contrast,  a Doom  like  simulation  of  a banking  computer-based  training  program  is  juxtaposition  that 
fails.  Filipczak  (1997)  describes  a Doom  like  environment  where  the  main  character  must  catch  “clients”  running 
around  on  the  screen  and  shoot  monsters  (which  represent  clients’  problems)  (sec  figure  2). 
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Figure  1.  Tools  from  Investigating  Lake  Iluca. 

When  you  catch  a client,  you  arc  then  taken  to  a multiple  choice  question  screen  where  you  must  answer  a 
policy  question.  If  you  get  the  question  correct,  you  can  continue  playing.  If  you  don’t,  then  you  must  return  to  a 
“traditional  CBT  tutorial”  to  study  the  fact.  I must  confess  that  I have  not  seen  this,  and  have  only  read  its 
description.  But  if  I understand  this  correctly,  then  the  game  is  not  part  of  the  training,  and  the  training  is  not  part  of 
the  game.  The  game,  as  stated  in  this  online  article  “...rides  along  with  the  course  serving  two  functions:  It  relieves 
boredom  by  letting  students  take  a break  from  the  self-paced  instruction;  and  it  serves  as  a motivator/refreshcr, 
giving  students  an  immediate  reason  to  study  and  recall  the  information,  and  a reward  when  they  do.”  From  this 
description,  it  sounds  like  a classic  example  of  something  not  to  do.  It  is  “gamus  intcruptus.”  An  environment  such 
as  this  could  never  engender  flow  because  the  environment  would  be  constantly  interrupted  by  switching  between 
instruction,  game,  and  test.  It  is  not  that  it  is  impossible  to  do,  but  it  may  be  that  Doom  is  simply  an  inappropriate 
model  to  follow  for  the  training.  An  integrated  environment  could  engender  flow.  In  order  to  have  an  integrated 
environment,  one  must  consider  carefully  an  integration  of  the  content,  the  controls,  and  the  patterns  of  interaction. 
In  the  Doom  like  simulation  described,  there  arc  in  fact  three  patterns  of  interaction.  One  is  the  traditional  CBT,  the 
second  is  the  game,  and  the  third  is  the  multiple  choice  questions.  While  some  juxtaposition  might  keep  users  on 
task  and  alert,  this  is  simply  far  too  much  juxtaposition  of  content  and  patterns  of  interaction. 
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Figure  2.  Rendering  of  the  Doom-like  game  from  Training  Magazine . 


Building  an  Environment  that  uses  games  to  engender  Flow 

Attention  to  detail 

One  thing  seems  to  be  abundantly  clear  in  planning  and  building  these  types  of  environments:  neatness 
counts.  The  quality  of  the  multi-media  assets  such  as  images,  sounds,  and  animations,  are  a key  factor  in  getting 
people  interested  in  the  game,  and  interested  in  playing  the  game.  This  is  an  important  issue  to  be  considered  in  the 
design  and  development  of  educational  software.  Rather  than  “settling”  for  assets,  we  should  be  working  to  find 
appropriate,  quality  images  and  sounds  to  make  learning  environments  richer,  and  ultimately  more  meaningful  and 
enjoyable  experiences.  However,  it  is  important  to  note  that  while  attention  to  detail  is  important,  they  should  have 
a purpose  to  them.  As  one  participant  in  a previous  study  stated,  “I  don’t  know  why  they  put  those  (video  clips)  in 
there.  Probably  because  they  could.”  Some  features  were  included  in  games  (hat  while  technically  impressive,  had 
no  real  relationship  to  the  environment  created.  This  is  true  in  many  pieces  of  educational  software  as  well. 
Pushing  the  envelope  is  a noble  ambition  (Jones,  Farquhar,  & Surry,  1995),  but  pushing  the  envelope  should  be  done 
relative  to  the  environment  itself,  and  not  simply  because  it  is  possible.  Figure  3 shows  an  example  of  a Main  Menu 
within  a learning  environment  on  the  saxophone.  The  environment  itself  is  striving  for  a “cool”  tone.  In  order  to 
draw  the  users  into  the  program,  it  plays  upon  the  learners  knowledge  of  the  saxophone  as  a jazz  instrument.  The 
program  seeks  to  establish  a tone  similar  to  that  of  a Jazz  Club.  The  main  menu  then  becomes  a “set  list,”  and  many 
of  the  other  controls  and  features  of  the  program  take  on  the  tone  of  “cool  jazz.”  From  background  graphics  and 
music  to  buttons,  having  this  supportive  theme  helps  carry  forward  the  theme.  Additionally,  it  creates  an 
environment  where  control  and  content  work  together. 
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Applying  Experiential  and  Reflective  Cognition 

Strategy  games  are  ones  in  which  the  user  must  employ  higher  order  thinking  skills  and  problem  solving 
skills  to  continue  playing  and  win  the  game.  This  supports  the  notion  of  reflective  cognition  set  forth  by  Norman 
(1993).  Twitch  games,  also  called  Thumb  Candy  (Herz,  1997),  are  games  in  which  the  user  must  react  quickly  to 
circumstances,  usually  by  killing  someone,  to  continue  playing  and  win  the  game,  SiniCity  and  War  Craft  II  arc 
immediate  examples  of  strategy  games,  while  games  like  Doom  are  consummate  twitch  games.  The  advantage  to  a 
twitch  game  is  that  the  movement  is  quick,  and  the  feedback  immediate.  This  works  to  keep  the  user  actively 
engaged.  However  the  level  of  this  engagement  is  often  superficial.  It  does  not  typically  engage  one  beyond  the 
most  basic  level  of  seeing,  pointing,  and  clicking.  Strategy  games  require  the  user  to  look  at  the  larger  problem,  and 
plan  a strategy  to  solve  the  problem.  In  some  games,  such  as  SimCity , the  results  of  your  decisions  arc  not 
immediately  recognized.  You  must  have  a fair  amount  of  internal  motivation  to  stay  with  the  game  to  realize  the 
fruits  of  your  labor.  While  twitch  games  offer  immediate  results  of  your  work,  strategy  games  appear  to  offer  a 
greater  feeling  of  accomplishment  and  satisfaction.  One  participant  who  was  playing  War  Craft  praised  the 
combination  of  “twitch  and  strategy.”  While  they  ultimately  liked  working  on  complex  problems  in  an 
environment,  they  also  appreciated  the  sheer  visceral  rush  of  immediate  feedback.  One  manifestation  of  this  in  a 
CBLE  might  be  seen  in  Figure  4.  In  a program  dealing  with  primary  colors  for  pre-readers,  users  atcract  with  the 
content  by  clicking  and  dragging,  one  of  the  most  common  methods  of  interaction  on  the  computer.  They  arc  given 
paint  brushes  to  drag  on  to  a picture.  If  the  correct  brushes  arc  used  to  create  the  correct  color  combination,  then  the 
paintbrushes  change  color,  and  the  learner  “paints”  the  picture  the  correct  color.  The  controls  in  this  exercise  arc 
indistinguishable  from  the  content  of  the  program,  and  the  method  of  interaction,  though  common,  serves  to  engage 
the  learner  both  physically  and  intellectually. 


From  Games  to  Flow 

Returning  to  the  table  that  relates  elements  of  How  to  manifestations  in  a computer  game,  what  follows  is 
an  effort  to  show  possible  manifestations  of  flow  activities  in  a CBLE. 


Conclusions 

Simply  put,  when  an  individual  is  in  a state  of  flow,  they  lose  themselves.  When  an  individual  is  engaged 
in  a CBLE,  they  arc  engaged  in  the  process  of  learning.  It  is  conceivable  that  attending  to  the  visual  and  aesthetic 
sensibilities  of  an  environment  can  make  it  more  engaging.  It  has  been  suggested  here  that  certain  aspects  of  the 
How  experience  can  be  manifested  in  CBLE.  By  looking  to  computer  games,  which  are  engaging  computer-based 
environments,  one  can  possibly  ferret  out  principles  for  engagement.  Suggestions  for  incorporating  gaming 
practices  have  been  set  forth  by  Jones  (1997).  The  idea  of  combining  flow  theory  with  the  design  of  computer- 
based  learning  environments  is  an  area  that  is  should  be  considered  for  further  study. 
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Table  2.  Relationship  between  Flow , games , and  design  ofCBLE's. 


Element  of  Flow 

Manifestation  in  a game 

Possible  manifestation  in  a 
CBLE 

1 . Task  that  we  can 
complete 

The  use  of  levels  in  a game  provide 
small  sections  that  lead  to  the 
completion  of  the  entire  task 

Exercises  relevant  to  the  content 
area  that  provide  “drill  and 
practice”  types  of  activities 
might  provide  learners  with 
areas  of  skill  to  focus  on.  Also 
aids  in  reinforcement. 

2.  Ability  to  concentrate 
on  task 

Creation  of  convincing  worlds  that 
draw  users  in.  The  Dungeons  and 
Labyrinths  in  Doom  II  help  you 
suspend  your  belief  systems  for  a time. 

Creation  of  seamless  integration 
of  tools,  tasks,  and  presentation 
of  information. 

3.  Task  has  clear  goals 

Survival,  collection  of  points, 
gathering  of  objects  and  artifacts, 
solving  the  puzzle 

Provide  problems  within  a 
learning  environment.  Despite 
the  desire  to  have  learners 
determine  their  own  problems, 
it  can  help  to  have  initial 
problems  identified  for  them. 

4.  Task  provides 

immediate  feedback 

Shoot  people  and  they  die.  Find  a clue, 
and  you  can  put  it  in  your  bag. 

Combining  appropriate  tools 
with  the  software  can  provide 
users  with  mechanisms  for 
meeting  goals  and  gaining 
feedback  from  the  software  (See 
Figure  1). 

5.  Deep  but  effortless 
involvement  (losing 
awareness  of  worry 
and  frustration  of 
everyday) 

The  creation  of  environments  that  are 
far  removed  from  what  we  know  to  be 
real  helps  suspend  belief  systems  and 
take  one  away  from  the  ordinary 

Keeping  the  visual  appearance 
of  the  environment  can  maintain 
consistency  which  may  help 
keep  users  focused  on  the  task 
at  hand.  Less  environmental 
juxtaposition  may  help  keep 
users  focused  (Sec  Figure  3). 

6.  Exercising  a sense  of 
control  over  their 
actions 

Mastering  the  controls  of  the  game, 
such  as  mouse  movement  of  keyboard 
combinations 

Providing  for  more  direct 
control  by  the  learner.  More 
learner  directed  movement  than 
designer  determined  paths. 

7.  Concern  for  self 
disappears  during 
flow,  but  sense  of  self 
is  stronger  after  flow 
activity 

Many  games  provide  for  an 
environment  that  is  a simulation  of  life 
and  death.  One  can  cheat  death  and 
not  really  die.  People  stay  up  all  night 
to  play  these  games.  It  is  the  creation 
of  an  integration  of  presentation, 
problem,  and  control  over  the  system 
that  promotes  this. 

Tools  that  promote  self 
confidence  (achievable  goals, 
tasks  that  arc  level  appropriate) 
can  help  strengthen  sense  of  self 
and  help  generate  greater  df 
efficacy  (Sec  Figure  4). 

8.  Sense  of  duration  of 
time  is  altered. 

Years  can  be  played  out  in  hours. 
Battles  can  be  conducted  in  minutes. 

Chunking  of  information  can 
help  keep  people  moving. 
Closure  (Jones  Okey,  1995)  can 
keep  users  working. 
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Introduction 

This  is  not  a typical  research  paper.  This  is  a paper  about  research.  In  1994,  four  fairly  recent  doctoral 
graduates  of  the  university  of  Georgia  were  discussing  the  possibilities  of  publishing  on  the  World  Wide  Web.  We 
established  Instructional  Research  Online  (InTRO)  as  a repository  of  research  papers,  media,  and  other  artifacts 
related  to  research  in  Instructional  Technology.  Defining  a direction  for  InTRO  was,  and  remains  an  iterative 
process.  The  website  (available:  http://intro.basc.org)  includes  sections  on  reviews  of  books  and  media,  original 
research,  links  to  resources  in  instructional  technology,  and  interviews  with  research  leaders.  One  feature  that  has 
helped  define  InTRO  and  its  purpose  has  been  the  section  on  interviews  with  research  leaders. 

These  interviews  arc  informal  conversations  with  people  who  conduct  research  in  the  field  of  instructional 
technology.  The  purpose  of  the  interviews  is  to  gain  insight  into  the  processes  and  issues  of  research  in  the  field  of 
instructional  technology  by  conversing  with  people  who  have  significant  experience  engaging  in  research  in  and 
tangential  to  our  field. 

Our  practice  of  interviewing  research  leaders  may  have  grown  out  of  a doctoral  seminar  in  research  at  the 
University  of  Georgia  taught  by  Tom  Reeves.  The  culminating  assignment  of  the  course  was  to  focus  on  a research 
leader  in  instructional  technology  and  to  read  from  that  person’s  body  of  research.  Analyzing  this  body  of  work, 
you  could  begin  to  sec  what  somebody’s  research  agenda  was,  how  their  research  had  remained  focused  on  a single 
area,  shifted  slightly,  or  shifted  radically.  The  final  phase  of  this  assignment  was  to  contact  this  person  and 
interview  them.  All  of  the  information  was  assimilated  and  presented  back  to  the  class. 

The  practice  of  interviewing  research  leaders  was  a valuable  one.  Wc  may  have  learned  as  much  from 
talking  to  these  people  personally  as  we  did  by  reading  journal  articles  authored  by  them.  Wc  certainly  came  to 
realize  that  people  who  arc  regarded  as  leaders  in  research  in  our  field  can  also  be  pleasant  people  who  arc  interested 
and  willing  to  talk  about  their  work.  Much  of  what  was  learned  about  research  in  this  class  was  through  contact 
with  people  on  a personal  level.  One  reason  for  this  may  he  that  much  of  what  people  know  is  not  set  forth  in  print. 
It  is  informal  knowledge,  or  patterns  of  understanding  that  may  not  be  documented,  codified,  or  shared  formally. 

Most  of  us  will  recognize  that  there  arc  times  when  wc  learn  as  much  working  on  projects  as  we  learn 
sitting  in  classes.  For  example,  while  one  may  study  front  end  analysis  it  is  not  until  one  docs  an  actual  analysis  that 
one  realizes  the  practical  application  of  gathering  information  about  an  instructional  environment.  Work  in  practical 
settings  helps  inform,  reinforce,  and  extend  what  wc  study.  One  reason  for  this  is  that  when  wc  work  on  projects  wc 
work  with  people  who  know  more  than  wc  do,  and  wc  arc  able  to  tap  into  the  informal  body  of  knowledge  that  they 
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hold.  This  is  true  in  the  study  of  research  as  well.  Informal  conversations  about  research  are  important  in  helping 
form  opinions  on  research,  and  in  helping  people  figure  out  how  to  conduct  research. 

In  our  interviews  we  were  interested  not  only  in  what  kind  of  research  these  people  were  doing,  but  we 
were  also  interested  in  what  advice  they  might  have  to  people  in  the  field  interested  in  conducting  research.  It 
remains  our  contention  that  there  is  much  to  be  learned  from  people  who  conduct  research.  Fittingly  enough,  our 
first  research  leader  interview  was  with  Tom  Reeves. 

Selection  of  Leaders 

In  every  field  there  arc  individuals  who  are  regarded  as  leaders  or,  if  you  will,  bellwethers  for  the  field. 
Their  wo:k  is  generally  recognized  as  important  because  it  not  only  informs  the  field,  but  it  also  shapes  the  direction 
of  the  field.  Their  work  may  not  always  be  agreed  with,  but  it  is  recognized  as  important  and  significant.  It  helps  to 
challenge  what  we  know  in  order  that  we  may  ultimately  know  more. 

The  selection  of  leaders  for  our  interviews  is  an  ongoing  process;  we  continue  to  conduct  and  publish 
interviews.  Our  original  selection  of  leaders  to  interview  was  certainly  based  on  their  contributions  to  the  field,  but 
other  factors  may  have  played  a role  in  their  selection  as  well.  The  InTRO  editorial  board  started  as  a team  of  young 
researchers.  While  we  did  the  initial  screening  without  any  formal  selection  criteria,  we  did  work  to  select  a balance 
of  perspectives.  It  was  and  remains  a goal  to  include  people  from  different  institutions  and  different  research  areas. 
We  have  included  not  only  people  conducting  research  in  academe,  but  also  people  conducting  research  in  industry. 

We  tended  to  interview  people  who  had  a significant  number  of  publications,  and  who  had  been  in  the  field 
for  more  than  10  years.  We  looked  to  people  who  were  active  in  professional  organizations  such  as  AECT,  and  we 
have  striven  to  include  a diversity  of  views  from  a diverse  group  of  people.  Given  that  we  all  have  some  connection 
to  UGA  (Susan  Land,  our  most  recent  editor  graduated  from  Florida  Stale  University  but  did  a postdoc  at  UGA)  our 
sample  may  be  biased  towards  that  institution.  Additionally,  there  is  a fair  amount  of  self  selection  among  our 
interviewees  as  a number  of  our  requests  for  interviews  have  been  declined.  Consequently  the  people  represented  in 
these  interviews  do  represent  good,  interesting,  and  challenging  perspectives,  though  they  may  not  represent  every 
perspective  at  this  point.  It  should  be  stated  that  the  InTRO  editorial  board  is  always  open  to  suggestions  on 
research  leaders  to  interview. 

Analyzing  the  Interviews 

The  purpose  of  this  paper  is  to  identify  and  report  on  an  analysis  of  the  published  interviews.  Interviews 
arc  conducted  via  email.  Typically,  we  review  a researcher's  work,  and  then  email  them  a set  of  questions  based  on 
that  review  that  we  think  will  be  of  interest  to  our  audience.  Once  we  receive  the  researcher's  response  we  then  send 
a set  of  follow-up  questions.  We  compile  these  with  the  original  responses,  ask  the  interviewee  to  review  it.  and 
then  post  it  on  InTRO. 

The  following  analysis  represents  a unique  opportunity  for  the  reader.  All  of  the  data  we  have  available  to 
us  is  available  to  you.  This  analysis  represents  what  was  said  during  the  course  of  the  interviews,  and  unlike  other 
types  of  interviews  used  in  research,  we  were  not  able  to  go  back  and  ask  for  clarification  after  analysis  had  begun. 
This  is  not,  in  the  strictest  sense,  a research  article.  It  is  a synthesis  of  comments,  suggestions,  and  other  important 
points  made  by  the  interviewees.  We  make  this  distinction  because  we  feel  that  there  arc  conclusions  that  we  think 
arc  appropriate,  hut  are  simply  not  borne  out  in  the  interviews.  There  are  conclusions  that  could  be  drawn  in  this 
paper  based  on  personal  experience  and  the  interviews,  but  we  shall  refrain  from  making  those  in  this  paper.  We 
wish  to  concentrate  on  setting  forth  only  what  can  be  drawn  from  the  interviews  themseives.  There  arc  obviously 
many  interpretations  that  are  possible  based  on  the  interviews.  This  pap^  and  the  accompanying  presentation 
represents  one.  We  invite  you  to  draw  others  for  yourselves.  And  with  that  invitation  we  also  extend  to  you  the 
invitation  to  formalize  your  interpretations  and  submit  them  to  InTRO  lor  publication. 

How  to  identify  a problem:  A question  of  questions 

The  journey  of  a thousand  miles  is  begun  with  a single  step.  This  may  be  true,  but  if  you  are  unclear  on  the 
direction  in  which  your  first  step  should  be,  your  journey  may  not  go  that  well.  This  is  the  problem  many  people 
have  in  conducting  research:  it  is  hard  to  identify  a unique  question.  The  interviews  provide  some  help  in  this  area. 
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Gaining  confidence 

Nearly  everyone  working  on  research  issues  has  faced  the  problem  of  identifying  an  area  in  which  to  do 
research.  There  is  no  shortage  of  important  questions  to  ask,  or  important  issues  to  study  according  to  the 
interviewees,  but  they  do  recognize  that  people  find  it  hard  to  gel  started  in  research.  Most  people  who  find  it  hard 
to  identify  a problem  may  not  suffer  from  a lack  of  ideas,  but  rather  from  a lack  of  confidence  that  their  ideas  were 
good  and  viable.  It  is  suggested  by  our  researchers  that  one  thing  that  has  made  them  “good”  at  research  has  been 
believing  that  their  ideas  were  important  and  of  interest.  They  have  learned  to  analyze  questions,  and  to  “trust  their 
instincts”  when  looking  at  problems  in  the  field.  One  thing  that  has  helped  them  “trust  their  instincts”  has  been  the 
involvement  of  a good  mentor  or  good  colleagues  who  can  serve  as  a sounding  board  for  idea.  Mentors  and 
colleagues  can  help  people  gain  support  for  their  ideas  and  give  them  the  encouragement  they  need  to  pursue  them. 

Reviewing  the  literature 

There  arc  many  ways  that  a researcher  can  gain  support  for  their  ideas.  One  is  by  careful  consideration  of 
the  existing  research.  One  sound  and  practical  method  suggested  on  how  to  identify  a research  area  is  to  think  about 
the  areas  that  you  want  to  know  something  about  and  go  read  about  them.  As  you  read  you  may  find  “holes”  in  the 
literature,  that  is  to  say  areas  in  the  literature  where  significant  questions  are  going  unanswered.  If  you  want  an 
answer  to  a question,  but  there  exists  no  answer  in  the  literature,  then  that  is  an  area  of  research  to  pursue.  The 
“holes”  are  where  you  should  be  working.  Reading  and  understanding  the  literature  is  critically  important,  as  is  the 
ability  to  read  and  synthesize  large  bodies  of  research.  One  researcher  feels  that  the  work  they  have  done  in  writing 
literature  reviews  is  the  most  difficult,  but  possibly  the  most  important  work  that  they  have  done. 

Finding  studies  and  other  articles  that  support  your  research  ideas  is  only  one  reason  to  consider  the 
literature  carefully.  Finding  absolutely  no  consideration  of  your  ideas  in  the  literature  is  a fairly  compelling 
indication  that  a pressing  need  for  your  work  exists.  Again,  this  is  looking  at  the  holes  in  the  literature  and  focusing 
your  work  to  fill  those  holes.  As  you  begin  to  do  research,  your  work  begins  to  help  answer  important  questions,  but 
may  not  be  able  to  answer  every  question.  We  are  reminded  by  many  research  leaders  that  few  individuals  will 
make  a difference  with  one  piece  of  work.  Collectively  we  conuibute  to  the  body  of  literature  in  the  field.  The 
knowledge  base  grows  as  more  people  ask  and  answer  questions.  The  combined  total  of  all  of  this  asking  and 
answering  is  what  we  know  in  the  field  of  instructional  technology.  It  is  contributing  to  this  body  of  knowledge  that 
is  important. 

Finding  questions  in  real  world  environments 

Still,  many  people  are  simply  trying  to  get  an  idea  started.  Reading  the  literature  and  finding  holes  in  the 
problem  may  seem  overwhelming  to  some.  Beyond  being  overwhelming,  it  may  not  be  practical  to  gain  all 
questions  from  the  literature.  Looking  at  naturally  occurring  environments  is  one  way  to  find  out  what  questions  arc 
out  there.  One  piece  of  advice  is  for  researchers  to  go  into  educational  environments,  such  as  schools,  and  observe 
people  using  an  educational  product.  We  should  look  to  sec  where  they  are  having  problems,  what  is  making  sense 
to  the  user,  and  what  is  confounding  the  user.  Noting  these  problems  can  provide  excellent  sources  for  ideas  in 
research. 

Combining  observations  and  literature  can  help  inform  your  reading,  and  help  to  isolate  problems  that 
literature  can  then  be  associated  with.  It  is  not  only  the  observations  of  people  using  other  people’s  products  that 
can  lead  to  interesting  research  projects,  but  it  is  also  observing  people  working  with  products  that  we  develop  that 
can  inform  research  questions. 

Instructional  technologists  are  involved  in  a number  of  efforts.  In  addition  to  reading,  we  may  be  working 
on  media  design  and  development  projects.  Practical  experience,  especially  for  people  new  to  the  field,  provide  us 
with  activities  that  enable  us  to  gain  a better  insight  in  to  the  way  the  field  works,  and  what  it  means  to  work  in  the 
field.  Many  of  the  researchers  interviewed  here  point  to  the  close  bond  between  research  and  development. 
Problems  encountered  during  a development  project  often  lead  to  interesting  research  questions.  Leaders  who  have 
worked  in  industry  state  that  their  experience  in  project  work  has  been  the  driving  force  in  many  of  their  research 
activities.  This  points  towards  the  important  relationship  between  research  and  development. 

The  relationship  between  research  and  development 

Instructional  Technology  is  an  active  field,  and  it  is  important  to  remember  that  in  the  final  analysis  it  is  our 
job  to  make  a positive  impact  in  the  way  that  people  learn.  Saettler  (1990)  reminds  us  that  technology  in  our  field  is 
defined  quite  broadly.  Wc  work  with  hard  technologies,  sometimes  referred  to  as  “things  that  plug  in,”  but  we  also 
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work  with  soft  technologies,  or  “advances  in  thought.”  Always,  we  are  looking  forward.  Though  much  of  what  wc 
do  is  practical  in  nature,  even  studies  in  seemingly  esoteric  areas  such  as  post  modernism  arc  used  to  inform  our 
work.  In  our  field,  research  certainly  influences  practice.  However,  this  is  a reciprocal  arrangement  because  often 
times  in  instructional  technology  practice  influences  research. 


Blurring  lines 

Most  research  leaders  find  that  the  boundaries  between  work  interests  and  research  interests  blur 
significantly.  Many  researchers  with  advanced  degrees  have  worked  for  years  before  returning  to  study  for  an 
advanced  degree  and  pursuing  a career  that  involves  research.  Because  of  this,  many  of  their  initial  research 
questions  cam  from  their  work  experiences.  As  they  continued  to  do  research,  they  also  continued  to  do  work  in  the 
field.  Some  of  that  work  may  have  been  in  the  design  and/or  the  development  of  educational  programs.  For  many 
of  our  interviewees,  they  were  involved  in  significant  evaluation  activities.  While  the  type  of  work  may  vary,  what 
remains  constant  is  the  connection  between  work  experiences  and  research  projects.  Many  researchers  may  serve 
dual  roles.  They  may  design  the  environment  conduct  research  on  both  the  process  of  building  the  environment  and 
the  ultimate  value  of  the  finished  environment.  This  indicates  a very  strong  connection  between  research  and 
development,  and  suggests  that  it  is  not  only  common  for  researchers  in  instructional  technology  to  also  be  designers 
and  developers  of  instructional  products,  but  that  it  is  also  highly  desirable. 

One  researcher  recommended  that  working  on  projects  was  so  important  that  young  academics  should  do  it 
for  free  if  need  be.  The  practical  applications  of  this  work  can  greatly  improve  your  skills  as  a researcher  and  keep 
you  in  touch  with  what  the  field  is  doing.  So  while  most  leaders  in  research  tend  to  credit  the  real  life  experience 
they  have,  they  recognize  it  as  not  only  important  in  giving  them  experience,  but  in  keeping  them  current  as  well. 

Connecting  research  and  practice 

Combing  research  and  development  may  be  desirable,  but  what  can  be  difficult  is  making  the  results  of 
your  research  valuable  to  others.  We  face  a problem  with  research  in  our  field  according  to  our  researchers.  Chiefly 
we  don’t  connect  practitioners  to  research  as  seamlessly  as  we  might  desire.  It  is  often  assumed  that  the  role  of  a 
researcher  is  to  tell  the  practitioner  what  to  do.  However,  people  who  arc  working  in  the  field  everyday  have  many 
ideas  that  can  make  interesting  research  questions,  or  may  have  answers  to  many  significant  questions  in  our  field. 
In  instructional  technology,  many  active  researchers  are  also  practicing  professionals,  but  the  connection  between 
practice  and  research  should  be  made  stronger.  It  has  been  suggested  by  some  leaders  that  our  research  efforts 
should  be  focused  on  making  information  available  to  practitioners  in  schools,  industrial,  or  military  applications. 
Our  research,  and  the  results  of  our  research,  need  to  be  more  accessible  to  practitioners.  Our  research,  and  our 
field,  needs  to  make  a greater  impact  on  the  environments  in  which  we  work;  our  research  results  need  to  be  more 
practical. 


The  most  important  piece  of  work 

Finding  interesting  work  and  letting  it  guide  your  research  can  make  your  research  not  only  more  valuable 
to  the  field,  but  more  interesting  to  yourself  as  well.  Many  of  our  interviewees  arc  remembered  for  a “seminal 
work”  or  a particular  theory  or  line  of  inquiry  that  they  have  been  working  on  for  a period  of  time.  However  nearly 
all  of  them  expressed  that  the  most  interesting  thing  that  they  had  done  was  what  they  were  working  on  now.  This 
indicates  a passion  for  research,  and  a commitment  to  new  ideas.  It  would  be  hard  to  be  involved  in  a project  if  you 
were  not  interested  in  it.  Therefore  one  recommendation  made  is  that  you  work  to  find  projects  that  can  hold  your 
interest. 

Doing  research  on  your  work  is  a piece  of  advice  that  is  often  heeded  by  many  people.  Working  in  the 
field  is  both  professionally  and  personally  rewarding;  Because  many  of  the  researchers  point  out  that  wc  need  to  be 
studying  naturally  occurring  environments  to  sec  how  people  learn,  working  in  environments  becomes  even  more 
critical.  Finding  a naturally  occurring  environment  in  which  you  can  work  is  not  easy.  Establishing  yourself  by 
working  on  a project,  or  in  an  educational  environment  can  make  research  populations  available  to  you. 

Choosing  a mentor 

There  is  much  to  know  about  research.  There  is  the  problem  of  identifying  questions,  seeking  funding, 
managing  the  process,  and  ultimately  disseminating  your  work  and  writing  for  publication.  One  of  the  most 
important  things  a novice  researcher  can  do  is  to  seek  out  a mentor  to  help  them  learn  and  master  this  process.  The 


definition  of  mentor  here  is  not  limited  to  the  relationship  between  a student  and  a professor  in  graduate  school.  A 
novice  researcher  is  well  advised  to  seek  out  others  in  the  field  who  can  help  them  work  in  the  area  of  research. 
Many  leaders  advise  not  only  the  novice  to  search  out  mentors,  but  also  the  mentors  to  treat  their  apprentices  in  a 
collegial  manner. 

The  role  of  the  mentor 

Mentors  can  do  a lot  for  an  apprentice.  Beyond  simply  teaching  them  how  to  do  things,  they  can  introduce 
them  to  people  in  the  field,  or  involve  them  in  research  and  writing  projects.  Apprentices  gain  valuable  experience 
in  writing  for  publication  from  their  mentors.  But  ultimately  what  the  mentor  can  offer  to.  an  apprentice  is 
confidence,  and  confidence  is  one  of  the  most  important  skills  one  can  develop  as  a researcher.  To  submit  your 
work  to  the  rigors  of  the  referee  process,  one  needs  to  have  a bit  of  an  ego,  and  a thick  skin.  Mentors  can  help 
novice  researchers  develop  these  in  the  right  amount  and  appropriate  manner.  One  researcher  relates  a story  of  how 
they  were  “cooked”  good  by  a discussant  at  a conference.  The  presentation  went  abysmally,  but  ultimately  the 
paper  was  reworked  and  published.  The  research,  ideas,  and  the  paper  had  value,  but  that  value  wasn’t  recognized 
during  the  first  presentation  of  the  idea.  It  was  not  until  sometime  later  that  the  paper  made  it  into  publication.  The 
point  is  that  simply  because  one  person  does  not  like  or  agree  with  a research  idea  it  does  not  mean  that  the  idea 
should  not  be  pursued.  Mentors  can  help  apprentices  sec  this  and  gain  the  skills  and  confidence  they  need  to 
continue  working  when  an  idea  is  rejected. 

The  role  of  the  apprentice 

An  apprentice  can  offer  much  to  the  mentor  as  well.  Often  times,  it  is  an  apprentice  that  gets  their  mentor 
involved  in  a particular  project  or  line  of  research.  One  suggested  reason  for  this  is  that  mentors  work  with  more 
and  more  apprentices.  While  a mentor  may  have  spent  their  time  reading  and  writing  in  the  past,  more  and  more  of 
their  time  is  spent  working  with  more  and  more  people.  It  becomes  a challenge  to  stay  abreast  of  everything  that  is 
going  on  in  the  field.  Often  times  it  is  the  apprentice,  who  can  focus  their  efforts  and  energies  more  narrowly,  who 
can  point  the  mentor  in  a new  direction.  The  point  here  is  one  of  collcgiality.  It  is  generally  recognized  that  people 
working  together  can  bring  diverse  views,  skills,  and  ideas  to  a project.  The  same  is  true  of  the  relationship  between 
the  mentor  and  the  apprentice.  Care  should  be  taken  in  choosing  a mentor.  You  should  choose  a mentor  who  not 
only  publishes,  but  who  publishes  with  their  apprentices.  Novice  researchers,  or  people  in  their  first  academic 
position,  should  choose  a mentor  where  they  work  who  can  help  them  find  resources,  locate  outlets  for  publishing, 
and  help  them  learn  to  write  grants. 

Most  people  recognize  that  the  mentor  has  much  to  offer  the  apprentice.  It  is  suggested  here  that  the 
apprentice  also  has  much  to  offer  the  mentor.  This  reciprocal  relationship  promotes  not  only  a greater  understanding 
of  the  field  to  the  novice  researcher,  but  also  helps  keep  the  mentor  current  in  working  with  others  in  the  field. 

One  caveat-  one  of  our  interviewees  points  out  that  there  simply  may  not  be  enough  mentors  to  go  around. 
Mentoring  takes  a lot  of  time  and  entails  a special  relationship  between  the  faculty  and  students.  Given  the  fairly 
large  student  faculty  ratio  in  many  of  our  programs  it  may  not  be  possible  for  every  student  to  have  a mentor.  In 
fact,  it  may  be  the  case  that  if  your  are  fortunate  enough  to  have  a mento  then  you  are  fortunate  indeed. 

A question  of  methods 

The  issue  of  whether  to  do  qualitative  or  quantitative  research  is  one  that  most  researchers  are  faced  with. 
Historically,  much  of  the  research  that  was  done  in  education  in  general  and  instructional  technology  specifically, 
was  quantitative  in  its  planning  and  execution.  For  many  years,  faculty  supervising  research  were  trained  solely  in 
statistical  research  methods.  Today,  more  and  more  researchers  arc  looking  to  alternative  research  methodologies 
such  as  qualitative  research. 

Research  traditions 

Much  of  our  research  methodology  has  been  derived  from  the  hard  sciences.  There  arc  many  reasons  for 
this.  Historically  the  field  of  education  has  constructed  controlled  environments  in  which  to  test  single  or  multiple 
variables.  Most  often  these  were  contrived  environments,  and  if  you  look  at  research  in  the  field  of  instructional 
technology  you  will  see  that  most  of  the  research  in  the  past  several  years  still  follows  this  paradigm  (Reeves,  1995). 

Many  researchers  interviewed  here  state  that  their  initial  research  involved  isolating  single  variables  and 
working  with  them  in  a controlled  environment.  One  reason  for  this  may  be  that  many  of  our  research  leaders 
originally  were  psychologists,  or  educational  psychologists  trained  in  the  dominant  research  paradigm  of  the  times. 
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The  dominant  research  paradigm  for  many  years  has  been  quasi-cxperimental  studies.  People  researched  in  a 
manner  in  which  they  were  taught  to  research.  Some  leaders  suggest  that  much  of  their  initial  research  agenda, 
while  interesting  to  them,  was  also  driven  by  the  tenure  and  promotion  process.  A combination  of  history  and 
environment  has  driven  the  field  for  years  in  the  types  of  methods  and  research  tools  it  uses.  Currently,  most  leaders 
are  recognizing  that  there  arc  more  tools  available  to  researchers  today  and  are  seeking  to  use  these  tools  in  research. 

Mixed  methods 

A recurring  message  from  nearly  all  our  research  leaders  is  to  avoid  the  qualitative  versus  quantitative 
debate  entirely.  This  debate  has  been  characterized  as  “mindless”  by  some  people.  Our  research  history,  and  much 
of  our  current  research  recognizes  the  need  to  isolate  variables.  However  much  of  our  current  research  also  realizes 
that  variables  in  the  real  world  arc  not  really  controllable,  and  they  rarely  behave  like  the  variables  that  are  able  to  be 
controlled  in  the  world  of  physical  science.  Most  of  the  people  interviewed  here  work  with  the  philosophy  that  in 
order  to  answer  certain  questions  you  will  need  certain  sets  of  tools,  and  therefore  suggest  that  researchers  study  and 
develop  skills  in  both  qualitative  and  quantitative  research  methods.  When  an  environment  presents  itself  you  will 
then  be  ready  to  study  it  regardless  of  what  questions  may  need  to  be  asked,  or  what  tools  arc  needed  to  answer  the 
questions. 

Additionally,  some  of  the  leaders  interviewed  here  suggest  that  beyond  qualitative  and  quantitative  research 
methods,  there  exists  the  need  to  consider  issues  in  yet  more  divergent  research  paradigms.  Among  these  arc  the 
methods  associated  with  post  modernism,  semiotics,  and  historical  research.  Additionally  there  are  at  least  two 
researchers  who  make  the  argument  that  our  research  not  only  needs  to  he  more  responsive  to  practitioners,  but  that 
it  needs  to  be  more  responsive  to  society  as  well.  There  is  small,  but  growing  call  in  the  field  for  socially 
responsible  research  in  instructional  technology. 

Disseminating  research 

Research  is  done  to  inform  practice,  and  the  results  of  research  are  meant  to  be  shared  with  the  larger 
community.  Disseminating  research  begins  with  having  questions  that  need  answering,  and  culminates  in  the 
publication  of  that  research. 

The  research  agenda 

Traditionally  research  is  disseminated  in  two  formal  ways:  the  conference  presentation  and  the  journal 
article.  Informally  the  results  of  research  can  be  spread  through  informal  conversation  among  colleagues.  People 
need  to  share  the  results  of  their  research  so  that  they  can  not  only  make  sense  out  of  what  they  are  doing,  but  also  so 
that  may  gain  insight  into  the  next  phase  of  the  research,  or  the  next  study  in  a long  term  research  agenda.  Many 
people  speak  of  research  as  a pipeline.  You  conduct  work  on  an  idea  that  gets  proposed  to  a conference.  The  initial 
paper  is  done  and  presented  at  a conference.  Based  on  reactions  and  feedback  at  the  conference  the  paper  is 
reworked  for  submission  to  a refereed  journal.  While  you  arc  working  on  one  project,  you  should  be  considering 
other  issues  that  you  can  work  on  later.  Work  that  can  be  proposed  to  conferences  the  next  year  will  keep  you  busy 
after  one  piece  goes  in  to  print.  The  goal  is  to  keep  the  pipeline  full  at  all  times,  meaning  that  there  should  always 
be  a project  to  work  on.  For  those  in  a tenure  earning  position,  it  should  be  noted  that  the  tenure  process  is  too  short 
to  begin  work  in  distinctly  different  areas,  but  that  rather  you  should  focus  your  work  within  a common  framework. 
This  common  framework,  or  area  of  research  is  often  referred  to  as  a research  agenda. 

For  most  researchers,  the  research  agenda  will  start  with  the  dissertation  or  thesis  and  move  forward  from 
there.  While  many  researchers  can  and  do  make  dramatic  shifts  in  their  research  interests,  most  tend  to  work  in  a 
broad  area  such  as  evaluation,  computer-based  learning  environments,  electronic  performance  support  systems, 
cross  cultural  research,  or  the  processes  associated  with  instructional  design  to  name  but  a few.  The  key  point  is  that 
by  staking  out  an  area  in  which  to  work  a researcher  does  a couple  of  significant  things.  They  focus  on  key 
problems  and  become  recognized  as  a person  who  can  inform  this  small  area  of  the  field.  Taken  in  concert  with 
other  people  working  in  cither  similar  areas,  or  in  completely  different  areas,  a fabric  is  woven  that  constitutes  the 
body  of  research  in  the  field.  By  establishing  a research  agenda,  or  a line  of  inquiry,  a researcher  gets  to  the 
business  of  making  a contribution  hy  advancing  the  knowledge  of  the  field.  As  you  become  recognized  as  an  expert 
in  that  area,  you  can  begin  to  help  others  conduct  research  in  that  area,  and  you  can  begin  to  make  connections 
between  your  research  area  and  other  research  areas.  For  example  a person  with  significant  research  experience  in 
distance  education  could  team  up  with  a person  with  significant  experience  in  hypertext  to  conduct  research  in  web- 
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based  instruction.  T,,c  rwo  ;:rc as  inform  each  otl.er,  and  can  be  used  to  move  forward  into  a new  area.  The  two 
areas  do  not  represent  nr*:  re*  'ippurtenances  to  each  other,  but  rather  represent  unique  applications  of  divergent  areas. 

Writing  research 

People  in  the  “real  world”  are  interested  in  our  work,  but  they  find  it  hard  to  put  their  hands  on  the 
information  they  need  easily.  This  is  suggested  by  more  than  one  researcher.  Our  field  is  filled  with  three  letter 
acronyms  (TLA’s)  that  are  often  seen  as  cliquish  in  nature  and  make  it  sometimes  more  difficult  for  people  to  use 
the  results  of  our  research  than  wc  would  intend.  While  it  raises  the  question  of  “who  do  you  write  for,  it  may  also 
suggest  that  rather  than  saying  things  to  sound  intellectual,  we  should  simply  say  what  we  mean.  We  should  avoid 
confusing  language,  or,  as  the  bumper  sticker  says:  "Eschew  obfuscation." 

It  is  a recurring  theme  in  nearly  all  of  the  interviews  that  one  of  the  biggest  problems  we  face  as  a field  is  in 
having  our  work  make  a difference  in  the  life  of  the  individual,  and  that  one  of  the  ways  in  which  we  can  achieve 
this  is  through  the  dissemination  of  our  research.  Clarity  of  writing  is  stressed,  as  is  the  age-old  soul  of  wit,  brevity. 

Conferences  and  journals 

Conference  presentations  and  journal  articles  remain  the  two  most  common  ways  to  disseminate  research. 
While  not  mentioned  in  the  interviews  directly,  it  is  well  known  that  the  lag  time  between  writing  and  publication 
can  be  fatal  to  a new  idea:  by  the  time  the  new  idea  gets  published  the  idea  is  already  generally  recognized  as  being 
obsolete.  Most  of  us  will  have  to  begin  planning  what  our  conference  presentations  will  be  so  long  before  the 
conference  even  begins  that  some  ideas  may  be  found  to  be  inappropriate  by  the  time  the  conference  arrives. 

One  reason  that  it  can  take  so  long  for  research  results  to  be  published  is  the  referee  process.  It  does  take 
time  for  busy  people  to  read  their  colleagues’  work  carefully  and  make  decisions  on  its  appropriateness  for 
publication.  Web-based  publishing  has  been  mentioned  by  more  than  one  researcher  as  an  alternative  to  the 
traditional  paper  based  journals.  While  changing  the  medium  itself  is  not  likely  to  yield  dramatic  results,  it  is 
possible  that  it  could  speed  up  both  the  publication  speed  and  help  increase  access  to  the  published  research. 

Conclusions 

Hannafin  and  Hannafin  (1995)  warn  that  in  the  future  IT  might  "continue  its  emphasis  on  training 
practitioners,  but  will  be  unable  to  forge  its  own  destiny  and  advance  its  own  research."  (P.  320)  It  is  important  that 
we  pay  greater  attention  to  research  and  to  training  researchers  if  wc  are  to  avoid  this  somewhat  dire  consequence. 
Becoming  a researcher  is  not  the  same  thing  as  learning  to  do  research.  We  can  learn  research  methodology  in  our 
classes  on  quantitative  and  qualitative  analysis.  We  can  learn  to  do  literature  review's  through  research  seminars. 
But  knowing  these  things  does  not  guarantee  that  one  will  be  able  to  put  them  together  in  an  intelligent,  scholarly, 
and  creative  fashion.  Only  by  working  with  those  among  us  who  arc  already  doing  quality  research,  can  we  began  to 
discover  how  to  synthesize  these  elements  in  such  a fashion  as  to  do  quality  research  ourselves.  There  may  not  be 
enough  mentors  to  go  around.  But  we  hope  that  through  the  interviews  with  the  research  leaders  on  InTRO  wc  can 
assist  those  mentors  among  us  in  reaching  and  in  teaching  enough  of  us  to  make  a difference. 
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14th  Foundations  Symposium 

Continuing  The  Debates/Discussions  Of  The  Foundations  Of  Our  Field. 

Education  And  Community* 

J.  Randall  Koetting 
University  of  Nevada 

The  National  Film  Board  of  Canada  produced  a film  a few  years  ago  called  “The  Manufacturing  of 
Consent.”  This  film  was  about  Noam  Chomsky’s  text  of  the  same  title  and  had  to  do  with  the  mass  media  and  the 
presentation  of  public  information.  The  film  is  a very  nice  critique  of  our  conference.  We  three  presenters  have  one 
hour  and  Fifteen  minutes  to  present,  essentially,  three  different  views  regarding  the  foundations  of  the  nature  of  our 
field,  as  well  as  allow  for  your  participation.  The  key  word  is  “concision”.  The  reality  is,  concision  is  impossible 
when  presenting  complex  ideas  and  positions  on  the  nature  of  our  field.  But....  here  is  one  of  the  presentations. 

Foundations  symposia  in  the  past  have  always  tried  to  deal  with  the  larger  questions  concerning  educational 
technology,  schooling,  and  society.  These  symposia  have  taken  the  form  of  discussions  centered  on  the 
philosophical,  historical,  psychological,  and  sociological  foundations  of  our  work.  It  has  been  the  belief  of  those  of 
us  who  have  participated  in  these  symposia  that  foundations  disciplines  raise  substantive  questions  regarding  the 
nature  of  our  work.  This  notion  of  foundations  is  not  meant  to  suggest  meta-narratives  of  the  “right”  interpretations, 
representations,  or  approaches  to  our  work,  but  more  of  the  diversity  of  our  approaches  toward  examining 
educational  technology  in  what  we  see  to  be  a complex  social  system. 

In  the  brief  time  that  I have,  I would  like  to  explore  the  notion  of  schools  as  sites  of  democratic  struggle, 
schools  as  sites  of  political  struggle.  Political  struggle  here  means  having  to  do  with  competing  ideological 
positions.  Seeing  schools  as  political  institutions  is  not  a mainstream  understanding  of  the  educational  experience. 
But  if  we  recognize  schools  as  political  institutions,  we  can  begin  to  see  them  as  “sites  of  possibility”.  By  this  I 
mean  there  is  no  one  best  system  of  schooling,  and  schools  are  places  where  the  experience  of  community  can  begin 
to  take  place.  Issues  of  social  justice,  the  common  good,  equality,  freedom,  etc.,  are  part  of  the  American  ideology, 
the  view  that  America  has  of  itself.  This  ideology  is  the  foundation  of  what  it  means  to  build  community. 
Community  can  exist  only  within  a framework  where  these  values  arc  cherished. 

The  Need  for  Community 

In  talking  about  the  need  for  community,  I am  talking  about  building  community  within  the  schooling 
experience.  Foundations  Symposia  have  always  advocated  reading  texts  outside  the  mainstream  instructional 
technology  literature.  This  literature  presents  us  with  different  viewpoints  regarding  education  that  we  can  use  to 
ask  ourselves  questions  about  our  own  work  in  technology.  For  example,  when  I read  David  Purpel’s  essay  (1993) 
on  “Educational  Discourse  and  Global  Crisis:  What’s  a Teacher  To  Do?”,  he  presents  me  with  this  statement:  there 
is  a major  gap  between  "mainstream  educational  discourse  and  the  urgent  social,  political  and  moral  crises  of  our 
time"  (p.278).  What  does  this  mean?  Mainstream  educational  discourse  is  technical/instrumental  in  nature.  This 
means  that  the  language  we  use  in  talking  about  education  has  to  do  with  a “scientific”  rationality  of  cause/effcct, 
means-ends  relationships  and  it  focuses  on  predictability,  efficiency  and  control.  Mainstream  educational  discourse 
focuses  on  educational  reform  that  is  seen  as  fine-tuning  the  system,  and  hence  any  incremental  rearrangement  of 
school  practice  is  seen  as  substantive  change.  We  keep  fixing  the  existing  system-  c.g.,  teacher  wages,  class  size 
reductions,  adding  services  and  programs,  changing  curricula,  adding  technology,  etc.,  without  substantive  change  in 
what  school  is  about.  In  other  words,  school  is  teaching  content,  school  is  producing  citizens. 

Our  mainstream  educational  language  does  not  take  into  consideration  the  complex  sucial/  political/econoinic  reality 
, i.c.,  the  sociocultural  context.  We  have  no  open  forums  for  debate  concerning  this  complex  reality.  As  a society, 
within  the  public  sphere,  we  seem  to  have  lost  a working  language  and  a desire  to  engage  each  other  in  substantive 
(moral)  discourse  about  education.  Our  working  ideas  and  understandings  of  democracy  have  become  abstractions. 
Yet  not  engaging  this  world  leaves  our  discourse  incomplete,  uncritical,  simplistic.  If  we  are  to  build  "rear 
community  (as  opposed  to  "virtual"  communities)  we  will  need  a way  to  engage  each  other  in  moral  discourse.  This 


* An  expanded  version  of  this  paper  was  published  in  Review  Journal  of  Philosophy  and  Social  Science. 


will  allow  us  to  develop  a language  of  democracy.  We  can  begin  to  do  this  through  a radical,  critical  praxisWhat  is 
critical  praxis,  critical  pedagogy?  What  is  radical/critical  education?  Very  briefly,  when  I read  Henry  Giroux’s  text 
Disturbing  Pleasures:  Learning  Popular  Culture  (1992),  Giroux  states  that  radical/  critical  education  is  not  a specific 
"body  of  knowledge",  but  is  rather  a paiticular  kind  of  practice  and  a way  of  "questioning  received  institutions  and 
received  assumptions".  Critical  forms  of  education  are  "interdisciplinary  in  nature",  question  the  "fundamental 
categories  of  all  disciplines",  and  have  a "public  mission  of  making  society  more  democratic"  ( p.10).  Paulo  Freirc 
refers  to  critical  teaching  as  an  "unquiet  pedagogy"  (1)  that  acknowledges  student/tcacher  voice  in  the  learning 
process.  An  unquiet  pedagogy  fosters  and  provides  the  opportunity  for  democratic  schooling  through  confronting 
issues  of  social  justice,  the  common  good,  equality  before  the  law,  etc.  These  issues  involve  the  characteristics  that 
are  needed  to  work  toward  the  creation  of  community  (i.e.  social  justice,  the  common  good,  equality,  etc.).  Creating 
democratic  community  requires  a different  kind  of  discourse  (other  than  a technical  discourse),  a moral  discourse, 
which  is  a way  to  talk  about  what  community  is. 

Within  the  above  discussion  I am  looking  for  ways  for  us  to  engage  each  other  in  a dialogue  that  will  move 
us  into  areas  that  we  can  cxnlore  toward  building  community.  My  interest  is  in  building  community  where  wc  arc, 
which  is  particularly  true  for  me  as  an  educator  within  an  institutional  setting  that  at  times  seems  to  be  at  odds  with 
building  a democratic,  participatory  community.  The  notion  of  building  democratic,  participatory  community  is 
important  because,  historically,  our  beliefs  and  actions  toward  democracy  have  changed.  There  is  no  one  vision  of 
democracy:  visions  of  democracy  have  "competing  interests"  as  well  as  differing  notions  of  "social  justice";  and 
because  of  our  inability  to  engage  in  moral  dialogue  within  society  (especially  within  schools),  it  is  increasingly 
difficult  to  clarify,  debate  or  act  on  these  visions,  much  less  develop  historical  understanding.  Question:  do  wc,  as  a 
society,  engage  in  dialogue  regarding  the  larger  issues?  Is  there  a forum  for  such  dialogue?  Landon  Beyer  suggests 
“no”.  He  writes  in  the  essay  “Schooling  for  the  Culture  of  Democracy”  (1988)  that  the  inability  of  society  (and  the 
"schools")  to  provide  a forum  for  open  discussion/on-going  dialogue  results  from 

...a  loss  of  communities  within  which  such  discourse  can  become  meaningful  and  prompt  the  requisite 
social  action.  Surrounded  by  larger  institutional  structures  favoring  technization,  commodification,  and  the 
therapeutic  privatization  of  social  relations,  we  have  all  but  lost  a sense  of  the  collective  social  good  so 
necessary  for  discussion  of  democratic  ideas  (Beyer,  1988,  p.221). 

In  other  words,  the  collective  social  good  is  replaced  by  individual  pathologies,  individual  problems, 
separated  from  the  social  context.  The  idea  of  individuals  "coping"  with  society  masks,  and  Hows,  social  reality  to 
be  unproblematic.  Jean  Elshtain  agrees,  and  in  her  text  Democracy  on  Trial  (1995)  comments  on  the  loss  of 
communities: 

“We  arc  in  danger  of  losing  democratic  civil  society.  It  is  that  simple  and  that  dangerous,  springing  as  it 
does,  not  from  a generous  openness  to  sharp  disagreement  -democratic  feistiness-  but  from  a cynical  and 
resentful  closing  off  of  others  (p.xii).” 

By  civil  society,  Elshtain  means  the  "many  forms  of  community  and  association  that  dot  the  landscape  of  a 
democratic  culture"  (1995,  p.5).  She  includes  families,  churches,  volunteer  groups  providing  "assistance  to  the 
needy",  self-help  movements,  trade  unions,  etc.  It  is  in  these  groups  that  the  "democratic  ethos  and  spirits  of  citizens 
arc  made  manifest"  (p.5).  The  "closing  off  from  others”  creates  a void  in  these  "civic  spaces",  and  suggests  the  loss 
of  skills  and  facility  in  face-to-face  communication.  (2)  However,  schools  are  also  sites  where  this  discussion  can  be 
fostered.  Schools  are  public  forums  (face-to-face  communication  can  and  does  happen),  subject  to  public  debate 
and  dialogue.  To  take  away  this  public  forum,  we  take  away  the  public  discourse. 

What  happens  when  wc  take  away  this  public  forum?  A void  is  created  in  these  civic  spaces,  and  it  is  filled 
by  "someone".  Elshtain  (1995)  suggests  that 

once  a world  of  personal  responsibility  with  its  characteristic  virtues  and  marks  of  decency  (honor, 
friendship,  fidelity,  and  fairness)  is  ruptured  or  emptied,  what  rushes  in  to  take  its  place  is  politics  as  a 
"technology  of  power,"  in  Vaclav  Havel's  words  (p.88). 

Politics  as  a technology  of  power  means  lhat  ideological  forms  of  coercion  seem  legitimate  because  they 
are  unquestioned,  common-sensical,  taken-for-granted  views.  Its  effects  are  expressed  in  feelings  of 
disenfranchisement,  loss  of  a sense  of  being  in  control  of  one’s  life  situation,  and  even  a further  withdrawal  from  the 
public  sphere  into  the  private.  Thus,  how  can  we  engage  each  other  in  purposeful  dialoguc/communication 
regarding  the  possibilities  of  personal  or  social  change,  the  nature  of  our  experiences,  the  creating  of  visions  that 


lead  to  increased  democratic  forms  of  participation  in  society,  etc.,  if  our  public. forums  arc  fractured,  dysfunctional? 
In  other  words,  how  do  we  talk  with  each  other  in  democratic  places? 

The  Need  for  Moral  Discourse 

The  above  discussion  brings  me  hack  to  the  importance  of  moral  discourse  as  an  integral  necessity  of 
establishing  a sense  of  community.  As  1 staled  at  the  outset  of  this  essay,  Purpel  (1993)  sees  the  need  for  a moral 
discourse  within  society  generally,  but  within  education  specifically  because  of  the  dangerous  chasm  between 
’’mainstream  educational  discourse  and  the  urgent  social,  political,  and  moral  crises  of  our  time"  (p.278)  (3). 
Mainstream  public  debate  both  within  and  without  education  seems  to  center  on  technical  (as  opposed  to  critical) 
issues.  Technical,  instrumental  educational  discourse  focused  on  technical  issues  should  be  of  great  concern  to  all  of 
us  who  believe  education  offers  possibilities  for  social  change.  I am  not  arguing  that  education  is  the  prevailing 
force  for  fundamental  social  and  cultural  change.  But  I would  argue  that  schools  have  a role  to  play. 

Again,  contemporary  educational  discourse  is  technical  in  nature.  Technical  and  instrumental  forms  of 
educational  discourse  focus  on  "ameliorating"  the  existing  system,  i.c.,  seeing  the  system  as  something  to  be  fixed  or 
''fine-tuned"  but  not  something  to  be  transformed.  Yet  the  urgent  social,  political,  and  moral  crises  of  our  time 
demand  more  than  fixing/fine-tuning.  These  crises  demand  engaging  in  a moral  discourse.  Moral  discourse  for 
Purpel  means  not  only  "moral  analysis",  but  aiso  the  task  "must  include  forging  a moral  vision-  one  that  can  inform 
and  energize  our  political  will  and  educational  strategy"  (Purpel,  1993,  p.282).  I interpret  moral  vision  as  a utopian 
vision,  in  which  "things  could  be  different  than  what  they  are".  This  utopian  vision  provides  us  with  possibility.  In 
the  language  of  Giroux  (1989),  we  are  moving  from  the  language  of  critique  to  the  language  of  possibility.  Without 
critique  and  the  vision  of  possibility,  change  becomes  impossible.  Without  critique  of  existing  school  arrangements 
and  visions  of  what  school  can  be,  substantive  school  reform  is  impossible.  This  is  the  meaning  of  schools  being 
sites  of  possibility. 

Developing  a "moral  vision"  includes  confronting  "painful  and  anguishing  dimensions  of  current 
educational  practice"  (Purpel,  1993,  p.282).  These  practices  have  to  do  with  the  technical/instrumental  concerns  of 
schooling,  which  are  important  from  a school  management  position,  but  leave  untouched  the  more  substantive 
issues  of  school  reform.  Purpel  is  quite  specific  here  regarding  the  "painful  and  anguishing  dimensions  of  current 
educational  practice”  and  is  worth  quoting  at  length: 

Teachers  are  caught  up  in  a system  in  which  individual  achievement,  competition,  success  and 
aggressiveness  are  essential  and  central  elements.  It  is  a system  in  which  education  becomes  an  instrument 
in  legitimizing  and  defining  hierarchy;  in  which  schools  are  a site  where  people  arc  sorted,  graded, 
classified,  and  labeled,  hence  giving  credence  to  the  tacit  social  value  that  dignity  is  to  be  earned.  Teachers 
are  asked  to  prepare  students  differently-  some  are  to  be  given  the  encouragement  and  skills  to  be  leaders, 
whereas  others  are  taught  to  endure  their  indignities  quietly  and  proudly.  It  is  a system  that  helps  sustain 
and  legitimize  a society  reveling  in  consumerism,  jingoism,  hedonism,  greed,  and  hierarchy"  (p.282). 

Moral  vision  must  include  "the  problematics  and  limitations  of  the  various  reforms  and  critiques 
that  range  from  criticisms  of  teachers  working  conditions  and  ways  to  basic  curriculum  reform"  (p.282).  What  arc 
we  left  with?  What  is  an  alternative? 

Democratic  Schooling 

Purpel’s  critical  stance  outlined  above  is  embodied  within  the  following  position  on  practice  within 
democratic  schooling,  providing  further  meaning  to  the  notion  of  schools  being  sites  of  possibility.  In  the  ASCD 
publication  Democratic  Schools  (1995),  Michael  Apple  and  James  Beane  outline  "conditions  on  which  democracy 
depends"  and  how  these  conditions  have  become  the  concerns  of  democratic  schools: 

1.  The  open  flow  of  ideas,  regardless  of  their  popularity,  that  enables  people  to  be  as  fully  informed  as 

possible. 

2.  Faith  in  the  individual  and  collective  capacity  of  people  to  create  possibilities' for  resolving  problems. 

3.  The  use  of  critical  reflection  and  analysis  to  evaluate  ideas,  problems,  and  policies. 

4.  Concern  for  the  welfare  of  others  and  "the  common  good." 

5.  Concern  for  the  dignity  and  rights  of  individuals  and  minorities. 

6.  An  understanding  that  democracy  is  not  so  much  an  "ideal"  to  be  pursued  as  an  "idealized"  set  of  values 
that  we  must  live  and  that  must  guide  our  life  as  a people. 


225  QQy 


7.  The  organization  of  social  institutions  to  promote  and  extend  the  democratic  way  of  life.  (Apple  and 
Beane,  1995,  pp.6-7). 

Although  it  makes  sense  to  have  democratic  schools  with  democratic  practices  if  students  arc  to  be 
informed  of  such  a way  of  life  and  government,  and  there  is  an  educational  history  of  such  practices  as  well  as 
renewed  interest  in  the  practice  of  democratic  schooling  (4),  Apple  and  Beane  (1995)  state 

there  is  perhaps  no  more  problematic  concept  in  education  than  that  of  democratic  schools,  a concept  that 
some  consider  almost  an  oxymoron.  How  can  this  be  so?  Simply  put,  many  people  believe  that  democracy 
is  nothing  more  than  a form  of  federal  government  and  thus  docs  not  apply  to  schools  and  other  social 
institutions.  Many  also  believe  that  democracy  is  a right  of  adults,  not  of  young  people.  And  some  believe 
that  democracy  simply  cannot  work  in  schools”  (p.7). 

Deborah  Meier  (1995)  answers  those  who  question  whether  or  not  schools  can  be  democratic,  those  who 
question  whether  democracy  is  for  the  young.  She  argues  that  it's  not  a question  of  whether  or  not  we  can  educate 
children  well,  but  rather  "Do  we  want  to  do  it  badly  enough?"  (p.4).  Hence  it  will  take  active  public  interest  and 
commitment  to  re-new  the  public  discourse  on  education  within  the  framework  of  democracy  suggested  above. 

Meier  (1995)  states  "Schools  can  squelch  intelligence,  they  can  foster  intolerance  and  disrespect,  they 
affect  the  way  we  sec  ourselves  in  the  pecking  order”,  and  that  is  precisely  the  reason  we  cannot  abandon  public 
schools  (p.6).  In  public  education,  students  are  united  by  "right",  not  "privilege".  Rights  are  contested.  This  gets 
messy. 

Democracy  is  not  always  convenient,  and  rights  do  require  sorting  out.  Neither  equity,  civil  rights,  nor 
mutual  respect  for  the  ideas  of  others  are  always  the  winners  even  in  public  institutions  -far  from  it-  but 
public  schooling  shifts  the  odds  in  favor  of  such  democratic  principles  (Meier,  1 995,  p.7). 

Public  schools  offer  the  possibility  of  developing  a sense  of  community  (schools  as  sites  of  possibility). 
Public  schools  put  us  face-to-face  with  others  we  might  sec  as  "statistics  or  categories"  (the  disenfranchised, 
students  of  different  ethnic  background,  etc).  Schools  are  places  where  we  learn  how  to  conduct  ourselves  in  public 
("for  better  or  worse").  And  public  schools  (as  public  forums)  can  provide  the  opportunity  and  practice  for 

such  political  conversation  across  divisions  of  race,  class,  religion,  and  ideology.  It  is  often  in  the  clash  of 
irreconcilable  ideas  that  we  can  learn  how  to  test  or  revise  ideas,  or  invent  new  ones  (Meier,  1995,  p.7). 

"The  clash  of  irreconcilable  ideas"  is  part  of  dealing  with  personal  difference,  and  personal  difference 
always  makes  things  "complicated".  Meier  argues  that  dealing  with  the  complicated  is  what 

education  for  citizenship  is  all  about.  "Ideas  -the  way  we  organize  knowledge-  are  the  medium  of  exchange  in 
democratic  life,  just  as  money  is  in  the  markei  place"  (1995,  pp.7-8). 

Meier  rightly  suggests  that  "Schools  embody  the  dreams  we  have  for  our  children.  All  of  them.  These 
dreams  must  remain  public  property"  (1995.  p.Il).  Purpel  (1993)  acknowledges  the  fragmentation  and  isolation  that 
indicates  a crisis  in  community  (cf.  ENhtain,  and  Meier  above),  but  rejects  the  notion  of  a "common  knowledge"  that 
binds  people  together.  Rather  we  need  to  "forge  a greater  share  of  community  through  a common  moral  vision" 
(p.283).  This  is  what  Meier  refers  to  as  the  dreams  we  have  for  our  children  .that  must  remain  public  property.  I 
interpret  this  common  moral  vision  as  incorporating  the  American  ideology,  the  American  creed,  which  includes 
notions  of  the  common  good,  social  justice,  equality  of  opportunity,  freedom  from  discrimination,  etc.  We  have  a 
language  that  speaks  of  these  values  that  America  stands  for,  if  not  in  actuality.  In  this  sense,  diversity  and  unity  arc 
U oppositional,  but  held  together  by  a common  ideological  position.  Here  I am  talking  about  the  debates  over 
multicultural  education. 

For  Purpel  and  Shapiro  (1995),  the  public  discourse  surrounding  education  is  “...nothing  less  than  the 
question  of  what  kind  of  world  wc  live  in,  what  we  wish  it  to  become,  and  who  arc  the  innovators  who  may  favor  or 
obstruct  such  possibilities”  (p.  1 36).  They  argue  that  the  public  discourse  about  education,  in  relation  to  the  broader 
questions  regarding  culture  and  society,  usually  have  been  treated  with  "varying  degrees  of  obfuscation,  denial,  and 
mystification"  (p.  1 36).  This  has  resulted  in  some  asserting  education's  "moral  and  political  neutrality",  some 
arguing  the  importance  of  an  "economic  rationale  for  education  (the  'human  capital'  view  of  the  purpose  of 
schools),"  and  the  political  right’s  position  that  "education  act  as  a brake  on  the  moral  and  cultural  disintegration  of 
the  nation"  (p. 1 36). 
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An  important  discussion  needs  to  take  place  here  regarding  the  notion  of  structure  and  agency,  i.e.,  the 
social  context  and  individual  autonomy  within  that  context  (the  sociocultural  context).  While  the  current  political 
climate  suggests  (advocates?)  rugged  individualism  regarding  responsibility  for  the  conditions  of  one's  life, 
suggesting  that  individual  problems  arc  individual  pathologies,  a sociocultural  analysis  suggests  the  individual 
cannot  be  understood  outside  of  that  social  context.  For  example,  the  growth  of  poverty  in  the  United  States  and  the 
attendant  conditions  that  work  against  individuals  calls  into  question  the  very  cornerstone  of  the  American  ideology, 
achievement  and  the  notion  of  a meritocracy.  As  Purpel  and  Shapiro  state 

Whether  it  be  personal  happiness,  economic  well-being,  or  social  success  and  recognition,  everything  is 
supposedly  in  the  hands  of  the  individual.  The  pop-psychology  formulation  of  Take  responsibility  for  your 
own  life1  and  'You  can  make  it  if  you  really  try,1  he  says,  reinforces  in  new  ways  the  cornerstone  of 
American  ideology,  namely,  the  belief  in  meritocracy.  If  you  want  happiness,  you  will  get  it;  if  you  don't 
have  it,  you  have  only  yourself  to  blame"  (p.I37). 

There  are  contradictory/competing  ideologies  our  society  works  within.  Sociocultural  analysis  could 
broaden  the  above  discussion  by  raising  questions  about  the  ideologies,  such  as  the  "achievement  ideology"  and  the 
notion  of  a meritocracy,  or  call  upon  other  aspects  of  the  ideology,  e.g.,  the  notion  of  social  justice,  equality  of 
opportunity,  the  common  good  established  through  a sense  of  community,  etc.  (that  set  of  values  that  help 
democracy  flourish).  Socio-cultural  analysis  could  identify  or  raise  questions  regarding  the  ever-widening  gap 
between  rich  and  poor,  the  global  economy,  employment  opportunities,  minimum  wage  employment,  loss  of 
manufacturing  jobs,  etc.,  i.e.,  the  current  economic  issues  of  today.  This  could  allow  us  to  break  out  of  a position  of 
rugged  individualism,  and  recognize  the  interdependence  of  the  T and  the  we"  (community). 

Let  me  try  to  pull  this  discussion  together. 

Ideas  about  Democracy,  Moral  Discourse,  and  Educational  Practice:  Movement  Toward 
Community 

The  notion  of  democracy  can  provide  a foundation  of  values  as  well  as  an  orientation  for  our  work  in 
education.  These  values  revolve  around  issues  of  social  justice,  the  common  good,  equality,  freedom,  community. 
These  terms  embody  those  broad  problematic  areas  that  sometimes  go  by  the  name  of  the  "American  Creed",  or  the 
"American  Ideology". (5)  These  issues  are  problematic  because  they  are  not  fixed  in  their  meaning,  but  rather  open 
to  interpretation.  To  inquire  into  these  issues  requires  a commitment  to  engage  each  other  in  moral  discourse.  This 
discourse  must  also  be  open  to  critique.  To  act  democratically,  to  move  from  language  to  action,  would  require  that 
wc  be  guided  by,  to  be  informed  by,  that  set  of  values.  Again,  I am  talking  about  democracy  as  a set  of  values  (the 
American  ideology)  as  well  as  democracy  as  a site  of  struggle,  establishing  horizontal  ties  between  citizen  and 
citizen. 

These  values  must  be  more  than  abstract  principles,  more  than  intellectual  pursuits,  but  again,  they  inform 
our  work.  They  become  our  orientation  toward  how  wc  talk  about  community.  Thus,  there  needs  to  be  some 
involvement  with  the  "political"  notion  of  "civic  education".  (6) 

This  notion  of  democracy  cannot  be  a-historical.  As  Giroux  (1988)  states,  "Democracy  is  a 'site'  of  struggle 
and  as  social  practice  is  informed  by  competing  ideological  conceptions  of  power,  politics,  and  community"  (pp.28- 
29).  To  develop  a language  of  citizenship  and  democracy  requires  examination  of  the  "horizontal  tics  between 
citizen  and  citizen".  Here  we  are  concerned  with  the  notion  of  difference  wherein  the  demands,  cultures,  and  social 
relations  of  different  groups  must  be  recognized  as  part  of  understanding  what  it  means  to  be  a pluralistic  society  (cf 
Giroux,  1988,  p.30).  Thus  "difference  and  identity"  must  be  central  to  any  debate  regarding  democracy  and  politics. 
This  means  that  "theories  of  difference"  arc  not  only  concerned  with  representation  of  identities  (how  the  "Other"  is 
represented),  but  also  must  be  concerned  with  issues  regarding  relations  of  power  (Giroux,  1994,  p.58). 

Identity  here  must  be  seen  as  "...the  effect  of  social  struggles  between  different  communities  over  issues  of 
representation,  the  distribution  of  material  resources,  and  the  practice  of  social  justice"  (Giroux,  1994,  p.61).  This 
understanding  of  identity  exemplifies  what  is  meant  by  democracy  as  a site  of  struggle. 

What  this  means  for  education  is  that  it  is  not  enough  to  read  the  texts  of  the  "Other".  Wanting  to  know  the 
“Other”,  Hazel  Carby  writes  in  "The  Multicultural  Wars"  "...  cannot  replace  the  desire  to  challenge  existing 
frameworks  of  segregation".  She  pointedly  asks: 

Have  wc,  as  a society,  successfully  eliminated  the  need  for  achieving  integration  through  political  agitation 
for  civil  rights,  and  opted  instead  for  knowing  each  other  through  cultural  texts?  (Carby,  quoted  in  Giroux, 
1994,  p.59). 
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This  again  points  to  the  political  notion  of  citizenship  and  democracy  as  a site  of  struggle.  For  educators  this  means 
developing  that  set  of  values  mentioned  above  that  will  inform  our  work. 

The  challenge  for  educators  is  how  "to  expand  the  basis  for  dialogue  and  community  without  erasing  a 
politics  of  difference"  (Giroux,  1994,  p.59).  Weeks  states 

We  may  not  be  able  to  find,  indeed  we  should  not  seek,  a single  way  of  life  that  would  satisfy  us  all.  That 
does  not  mean  that  we  cannot  agree  on  common  political  ends:  the  construction  of  what  can  best  be 
described  as  a "community  of  communities",  to  achieve  a maximum  political  unity  without  denying 
difference  (quoted  in  Giroux,  1994,  p.59). 

I would  like  to  conclude  with  a final  comment  on  the  notion  of  democracy  being  a site  of  struggle.  Given 
the  complexity  of  our  society  today,  the  global  economy,  our  national  economic  outlook,  legislation  favoring 
corporate  interests,  minimum  wage  employment  fueled  by  a service  economy  and  the  demise  of  the  middle  class  in 
terms  of  living  wages  and  loss  of  industry,  the  rich  getting  richer  and  the  poor  getting  poorer  and  growing  in 
numbers,  the  backlash  against  multiculturalism  through  a discourse  of  what  Giroux  (1994)  calls  "imaginary  unities 
of  common  culture  and  national  identity"  (p.57;  also  cf  Schlesinger,  1993),  etc.,  these  complexities  exemplify  what 
is  meant  by  democracy  being  a site  of  struggle. 

Recognizing  that  text-based  media  can  become  the  basis  for  our  public  discourse  and  can  help  shape  that 
discourse,  we  cannot  confuse  our  creating  and  receiving  texts  through  our  keyboards  with  the  struggle  for 
democracy.  Democracy  is  fought  for  face  to  face.  Democracy  is  fought  for  over  specific  issues.  The  struggle  for 
democracy  is  local,  as  local  and  specific  as  our  workplace.  Schools  arc  sites  of  struggle,  sites  of  possibility.  As 
identified  above,  Apple  and  Beane’s  (1995)  “conditions  on  which  democracy  depends”  provide  a substantive 
theoretical  grounding  for  school  practice,  possibility. 

Finally,  it  is  in  community,  in  the  company  of  others,  where  our  sites  of  struggle  can  make  a critical 
difference.  And  as  Elshtain  (1995)  reminds  us, 

...democracy  is  for  the  stout  of  heart  who  know  there  arc  things  worth  fighting  for  in  a world  of  paradox, 
ambiguity,  and  irony.  This  democratic  way  -moderation  with  courage,  open  to  compromise  from  a basis  of 
principle-  is  the  rare  but  occasionally  attainable  fruit  of  the  democratic  imagination  and  the  democratic 
citizen  in  action.  Democracy  is  on  trial  in  our  lime,  beleaguered  by  foes  and  bedeviled  by  friends.  But 
there  have  always  been  skeptics  (pp. 89-90). 

As  educators,  we  cannot  afford  to  be  part  of  the  skeptics.  If  schools  really  arc  the  "conscious  embodiment  of  the 
way  we  want  our  next  generation  to  understand  their  world  and  their  place  in  it"  (Meier,  1995,  p.135),  it  is 
imperative  to  know  what  that  message/ vision  might  be.  And  our  generation  will  have  succeeded  in  not  permitting 
democracy  to  pass  away  as  in  a dream  (cf  opening  quote  to  this  chapter).  And  this  must  be  done  by  the  “real”  as 
well  as  the  “virtual”  attempts  at  public  education. 

Endnotes 

1.  Frcirc  uses  the  term  “unquiet”  to  refer  to  critical  and  radical  pedagogy:  “A  pedagogy  is  that  much  more  critical 
and  radical  the  more  investigative  and  less  certain  of  'certainties'  it  is.  The  more  unquiet  a pedagogy,  the  more 
critical  it  will  become”  (quoted  in  Kutz  and  Roskelly,  1991)  An  unquiet,  or  critical  pedagogy  confronts  our 
experiencing  of  the  social-school-life  world  and  acknowledges  the  possible  contradictions.  An  unquiet/critical 
pedagogy  is  informed  by  and  allows  for  the  technical,  practical  and  emancipatory  forms  of  knowing  within  the 
learning  process.  This  pedagogy  confronts  notions  of  cultural  capital,  achievement  ideology  and  social 
reproduction,  as  well  as  the  social  construction  of  knowledge  and  how  these  are  played  out  in  schooling. 

2.  Meier  (1995)  also  refers  to  the  loss  of  democratic  spaces:  "The  formal  and  informal  institutions  that  were  once 
accessible  to  the  majority  of  children  and  that  grounded  the  young  in  a society  of  responsible  adults  are  missing  for 
most,  at  precisely  the  moment  they  arc  most  needed.  Facc-lo-face  meeting  places  such  as  political  clubs,  union 
halls,  and  settlement  houses  have  all  hut  disappeared.  These  were  not  only  places  of  nurturancc,  but  places  where 
we  learned  skills,  felt  safe  enough  to  take  needed  risks,  learned  to  believe  in  the  future.  Only  schools  remain"  (pp.9- 
10). 

This  is  not  a romanticization  of  the  past,  but  a recognition  that,  for  better  or  worse,  our  public  spaces  were 
diverse  and  represented  the  possibility  for  initiation  and  commitment  into  public  life. 
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3.  Beyer  (1988)  argues  that  we  have  lost  our  ability  to  engage  each  other  in  purposive  dialogue  i.c.,  we  have  "lost 
the  ability  to  use  moral  language  sensibly".  Quoting  Alasdair  MacIntyre,  in  place  of  moral  discourse, 

what  we  possess  ...  arc  the  fragments  of  a conceptual  scheme,  parts  which  now  lack  those  contexts  from  which  their 
significance  derived.  We  possess  indeed  simulacra  of  morality,  we  continue  to  use  many  of  the  key  expressions. 
But  we  have  -very  largely,  if  not  entirely-  lost  our  comprehension,  both  theoretical  and  practical,  of  morality  (Beyer, 
1988,  p.223). 

Beyer  (1988)  attributes  the  inability  to  use  moral  language  sensibly  to  forms  of  academic  theorizing  that  have 
"weakened  the  authenticity  and  potency  of  moral  discourse"  (p.223).  Be  discusses  three  such  forms:  First, 
conceptual  analysis  in  philosophy  which  tends  to  separate  moral  debate  from  social  context.  This  has  to  do  with 
looking  at  “ideal  types”  which  have  the  essential  characteristics  of  the  issue  under  discussion  (c.g.  notions  of 
freedom,  social  justice,  elc.).  This  form  of  conceptual  analysis  looks  for  refinement  and  analysis  of  word 
usc/meaning  in  particular  situations  to  clarify  meaning.  The  end  is  to  be  more  precise  in  wlmt  terms  mean,  in  order 
to  identify  and  correct  ambiguous  or  incorrect  usage.  Secondly,  a position  of  "epistemological  and  ontological" 
dualism  with  a commitment  to  certitude  through  forms  of  empirical  investigation.  The  dualism  "inherent  in 
positivistic  endeavors  demeans  the  viability  of  moral  judgments"  (p.225).  Here  Beyer  is  referencing  the  over- 
reliance  on  positivistic  forms  of  inquiry  (empirical  science)  which  claims  objectivity,  value  neutrality,  and  an 
atheoretical  stance.  Within  such  a position  "knowledge  is  to  be  found  precisely  by  separating  our  observations  and 
analyses  from  that  untrustworthy  social  context  from  which,  as  Plato  surmised,  only  opinion  can  spring"  (Beyer, 
1988,  p.225).  And,  third,  the  university  research  community,  as  a community  of  scholars  involved  in  the 
"conservation,  development,  and  dissemination"  of  our  cultural  heritage,  shifted  their  allegiance,  becoming  more 
closely  allied  with  "social  and  governmental  agencies,  and  responsive  to  the  demands  of  the  growing  corporate 
sector".  This  had  the  effect  of  further  separating  moral  discourse  from  "the  production  of  knowledge",  which 
production  happened  within  increasingly  narrow  disciplines  and  sub-disciplines,  while  moral  inquiry  belonged  to 
one  area  of  academic  inquiry  (p.225),  namely  philosophy. 

4.  See,  for  example,  Apple  and  Beane,  1995;  Wood,  1992;  Meier,  1995;  Rose,  1995;  et  al. 

5.  The  American  ideology  can  mean  meritocracy,  rugged  individualism  and  independence,  etc.  Yet  there  arc  other 
meanings  as  well.  I include  that  strong  set  of  values  that  "America"  prides  itself  on;  social  justice,  the  common 
good  and  the  building  of  community,  equality  of  opportunity,  etc. 

6.  This  is  the  meaning  of  Apple  and  Beane's  (1995)  set  of  conditions  that  arc  part  of  democratic  practice,  especially 
the  sixth  one:  "An  understanding  that  democracy  is  no*  so  much  an  'ideal'  to  he  pursued  as  an  'idealized'  set  of  values 
that  we  must  live  and  that  must  guide  our  life  as  a people"  (p.7). 
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SYMPOSIUM 

Making  Systemic  Change  in  Education  Work:  Social  and  Historical 

Considerations 


Critical  Literacy,  School  Reform  and  Social  Transformation 

J.  Randal!  Koettinp, 

University  of  Nevada 


My  brief  comments  are  concerned  with  the  notion  of  critical  literacy,  school  reform  and  social 
transformation.  Much  emphasis  has  been  and  continues  to  be  placed  on  school  reform,  without  much  consideration 
or  analyses  of  the  social  conditions/issucs  that  affect  change  within  educational  settings.  My  main  position  is  that 
our  educational  reform  efforts  arc  doomed  to  failure  because  we  view  educational  reform/changc  in  isolation  from 
the  rest  of  the  system,  seeing  reform  as  technical  change. 

The  mainstream  discourse  of  education  over  the  past  sixteen  years  has  been  shaped  by  the  1982  publication 
of  “A  Nation  at  Risk”  and  the  1990  publication  of  “America  2000".  The  discussion  generated  from  these  reports 
has  to  do  with  fine-tuning  education  through  incremental,  technical  changes,  and  intimates  a causal  link  between  the 
state  of  the  economy  and  the  work  of  the  schools.  The  increasing  call  for  “National  Standards”  for  education, 
restructuring  of  schools,  vouchers,  privatization  of  schooling  are  part  of  the  same  search  for  a causal  link.  There  is 
also  the  assumption  that  “the  purpose  of  public  education  is  to  prepare  Americans  to  compete,  both  as  individuals 
and  as  a society,  in  the  new  global  economic  order”  (Sehr,  1997,  p.l).  It  is  argued  that  raising  standards,  making 
education  better,  we  will  have  a work  force  that  will  qualify  for  better  jobs,  higher  wages,  and  America  will  prosper. 
However,  searching/foeusing  on  this  causal  link  (in  isolation  from  the  larger  sociocultural  context)  serves  to  mask, 
or  divert  attention  away  from  the  larger  issues. 

What  are  some  of  the  larger  issues'?  The  increasing  division  between  the  haves/have  nols;  the  demise  of  the 
middle  class;  the  movement  from  industrial  jobs  to  the  proliferation  of  jobs  within  the  service  economy  (minimum 
wage  employment  vs  living  wage  employment  giving  rise  to  increasing  numbers  of  workers  being  classified  as  the 
working  poor);  increased  automation  and  downsizing  in  the  workplace;  one  of  the  highest  incarceration  rates  among 
industrialized  nations;  decay  of  our  urban  areas;  etc.  I conclude  from  this  that  our  main  discussion  should  not  be  on 
school  reform,  but  about  social  transformation.  The  major  problems  facing  our  society  today  have  to  do  with 
economic  realities  and  deep  social  issucs/problcms.  Schools  exist  within  the  larger  social  contcxt/systcm  (society). 
Schools  may  be  part  of  the  problem,  but  they  arc  not  the  problem. 

Schools  are  part  of  the  problem  when  their  efforts  at  change  resemble  fine-tuning  the  system  without 
acknowledging  the  larger  issues.  Take,  for  example,  the  push  toward  defining  excellence  as  measured  by  high 
standardized  test  scores.  Politicians,  reformers,  the  business  community,  have  nc‘  been  able  to  lake  the  discussions 
of  standards  beyond  students  scoring  well  on  tests.  They  have  not  been  asked,  nor  have  they  volunteered, 
information  as  to  what  they  believe  will  happen  within  our  communities  when  the  schools  raise  everyone  up  to  the 
90th  percentile.  What  affect  will  that  have  on  our  communities?  How  will  the  quality  of  life  within  our 
communities  change  when  wc  accomplish  this  feat?  Will  there  automatically  be  more  jobs  available  from  which  our 
students  can  chose?  Will  the  economy  take  a sudden  turn  for  the  better  in  anticipation  of  the  new,  informed  labor 
force?  1 contend  the  only  reality  known  with  any  certainty  about  raising  the  test  scores  is  that  the  test  companies 
will  change  the  tests,  because  the  tests  are  not  doing  what  they  are  supposed  to  do,  namely,  discriminating  students 
along  a normal  distribution. 

As  cynical  as  the  above  may  sound,  I believe  it  is  a valid  critique  of  what  wc  read  in  the  mainstream  press, 
and  what  all  too  much  of  the  literature  in  education  supports.  As  Shannon  suggests  (1993) 

...  the  illusion  that  schooling  is  an  economic  anchor  influences,  if  not  directly  a"  ts,  the  immediate 
circumstances  of  schooling  in  profound  ways.  For  example,  extended  school  days,  ‘smartened  up’ 
textbooks,  intensified  curricula,  and  new  calls  for  national  standards  and  assessments  arc  direct 
consequences  of  the  promotion  and  power  of  these  advocates  and  their  illusions.  Teachers  who  ignore  this 
connection  between  classroom  and  social  life  arc  likely  to  be  at  a disadvantage  when  trying  to  participate  in 
the  restructuring  of  schools”  (p.87). 
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Absent  from  this  mainstream  discourse  regarding  educational  reform  is  any  sense  of  schools  as  sites  of 
democratic  struggle  regarding  representation,  voice,  and  action.  Schools  as  sites  of  democratic  struggle  require 
participants  to  be  able  to  read  the  word  and  read  the  world.  Reading  the  word  and  the  world  necessitate  an 
understanding  of  literacy  that  centers  on  critical  readings  of  context  “to  expose  false  illusions”  projected  (the  causal 
links  mentioned  earlier),  “writing  to  set  the  record  straight”,  and  “action  to  demonstrate  that  schools  could  be  places 
where  teachers  and  students  develop  democratic  voices  in  order  to  struggle  against  the  realities  of  poverty  in 
America”  (Shannon,  1993,  p.87).  This  notion  of  literacy  is  grounded  in  the  language  of  critique  which  provides  a 
discourse  of  possibility  that  could  lead  to  substantive  change.  Hence,  in  order  to  have  meaningful  school  reform 
efforts  we  need  to  create  conditions  wherein  “...wc  can  develop  democratic  voices  at  all  levels  of  schooling  so  that 
together  wc  can  engage  in  an  active  public  life”  (Shannon,  1993,  p.90). 
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Meeting  Software  Needs  of  Public  and  Higher  Education:  A Collaborative 

Model 

Cheryl  Krawchuk 
University  of  Arkansas 


Abstract 

While  collaboration  between  public  schools  and  higher  education  is  prevalent  due  to  the  Professional 
Development  movements,  one  area  of  potential  collaboration  has  been  overlooked.  This  area  consists  of  the  need  in 
public  schools  for  educational  software  and  the  need  in  computer  education  for  senior  software  design  projects. 
This  paper  discusses  the  ongoing  prototype  of  the  Software  Collaboration  Model  designed  to  meet  the  needs  of  both 
public  and  higher  education.  The  model  is  based  on  instructional  design  principles  and  proposes  that  both  public 
and  higher  education  institutions  can  benefit  through  the  collaboration  of  senior  software  design  students  and  in- 
service  public  school  teachers.  Findings  from  the  initial  prototype  indicate  that  the  model  needs  to  be  modified, 
giving  more  attention  to  communication  between  all  involved  parties , instructional  design , and  allowing  for 
revisions  of  each  step  of  the  process.  Such  modifications  have  been  made  and  a revised  model  is  suggested. 

Rationale 

The  idea  to  develop  a model  for  collaboration  between  in-scrvicc  teachers  and  senior  computer  science 
majors  emanated  out  of  separate  informal  conversations  with  both  parties.  Although  the  conversations  varied  in 
perspective,  the  underlying  theme  (the  need  for  software  development)  was  the  same.  The  in-service  teachers 
wanted  software  developed  specifically  for  th  *ir  students’  needs  while  the  computer  science  students  needed  to 
develop  a high  quality  software  program.  Given  the  common  interest  in  software  development  and  the 
complimentary  nature  of  their  needs,  the  possibility  of  collaboration  arose. 

To  verify  the  potential  for  a collaborative  model,  it  was  necessary  to  investigate  the  needs  of  both  parties 
and  identify  the  benefits  of  a collaboration.  Hence,  an  informal  needs  assessment  consisting  of  individual  and  group 
interviews  and  a benefits  analysis  was  performed.  Both  in-service  teachers  and  senior  computer  science  design 
students  were  assessed  prior  to  the  creation  of  this  collaborative  model. 

In  assessing  the  technology  needs  for  public  school  teachers,  in-service  teachers  from  a rural  middle  school 
were  the  participants.  Subsequent  to  the  assessment,  one  obvious  area  of  need  that  emerged  was  for  relevant  and 
affordable  educational  software.  Most  teachers  indicated  a lack  of  funding  for  software,  software  that  is  relevant  to 
specific  curricular  needs,  or  both.  This  finding  supports  previous  literature  that  suggest  teachers  and  schools  lack 
content-suitable  software  and/or  funding  for  such  software  (Braswell,  1994;  Oughton  & Krawchuk,  1997). 

In  conjunction  with  the  need  for  content-suitable  software  came  the  perceived  need  to  improve  math  and 
science  scores  across  the  state.  The  teachers  indicated  that  standard  test  scores  were  below  acceptable  national  and 
slate  levels.  These  feelings  were  validated  recently  as  the  state  SAT-9  test  reported  that  over  60%  of  graduating 
seniors  did  not  pass  the  math  portion  of  the  test.  As  such,  during  the  interviews  math  was  identified  by  the  teachers 
as  a content  area  in  need  of  software  assistance. 

With  the  lack  of  funding  and  available  software,  teachers  identified  a third  area  of  need.  They  reported 
that,  although  they  arc  willing  to  use  supplemental  software  in  their  classrooms,  they  are  not  capable  of  creating 
appropriate  software  due  to  time  constraints  and/or  a lack  of  skills.  While  these  concerns  are  cogent,  the  teachers’ 
references  to  time  and  skills  are  not  new  and  have  been  identified  in  previous  research  as  areas  of  consideration 
(Golas,  1993;  Maddux,  1994;  Oughton  & Krawchuk,  1997).  In  summary,  the  findings  of  the  needs  assessment  on  in- 
service  teachers  at  a rural  middle  school  revealed  that  these  in-scrvicc  teachers  want  math-related  educational 
software  that  is  inexpensive  and  content  -relevant  without  having  to  create  the  time  or  skills  necessary  to  develop  it 
themselves. 

The  second  stage  of  the  needs  assessment  focused  on  senior  software  design  students  at  a mid-sized  public 
university.  When  assessing  the  senior  software  design  students,  two  areas  of  need  were  identified.  First,  students 
were  required  to  create  an  extensive  software  program  as  a final  project  in  order  to  graduate.  The  students  viewed 
this  area  as  “THE”  need.  They  fell  as  though  all  other  needs  were  secondary  due  to  the  fact  that  their  careers  were 
determined  by  this  one  project.  Thus,  their  primary  focus  was  to  complete  a high-quality  final  project  in  a manner 
that  allowed  for  their  graduation. 
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Secondly,  because  of  the  high  quality  of  these  software  projects,  several  students  were  interested  in 
marketing  their  products.  However,  they  felt  they  would  encounter  difficulty  when  trying  to  find  subjects  to  Beta 
test  their  software  products.  They  feared  that  they  would  be  unable  to  find  an  adequate  number  of  subjects  as  well 
as  subjects  that  were  appropriate  for  the  content.  To  summarize,  the  needs  of  the  software  design  students  included 
the  production  of  a high  quality  software  program  and  the  Beta  testing  of  that  program  by  adequate  and  appropriate 
subjects  before  marketing. 

After  examining  the  stated  needs  of  these  two  parties,  the  potential  benefits  of  a collaboration  became 
apparent.  The  teachers  could  benefit  from  the  collaboration  by  obtaining  free  and  relevant  math-based  software 
programs  for  their  students.  At  the  same  time,  the  software  design  students  could  benefit  by  meeting  their  degree 
requirements  (producing  a high  quality  program)  and  obtaining  a subject  pool  for  Beta  testing  of  their  program. 
Given  the  potential  win-win  outcome  from  a collaboration,  the  development  of  a collaborative  model  began. 

Participants 

Upon  completion  of  the  needs  assessment,  a team  of  software  design  students  and  an  in-service  teacher 
were  chosen  to  work  together  on  a software  development  project.  The  software  design  team  consisted  of  three 
senior  software  design  students.  They  were  required  to  collaboratively  create  a high  quality  software  program  as  a 
final  project  before  graduation.  They  were  also  very  interested  in  marketing  their  final  product  and  wanted  to  focus 
on  an  educational  program  due  to  a large  market  for  educational  software  programs.  Both  of  these  factors  acted  as 
strong  motivators  for  the  students  to  participate  in  this  project. 

The  decreasing  math  scores  and  need  for  educational  software  motivated  the  in-service  teacher  to 
participate.  She  teaches  5-8  grade  middle  school  mathematics  students  on  a daily  basis.  She  appeared  extremely 
interested  in  attaining  free  and  relevant  software  for  her  students  and  consistently  indicated  her  need  for  computer- 
assisted  instruction  for  her  students.  As  such,  she  immediately  became  actively  involved  in  this  project. 

Software  Collaboration  Model 

While  collaboration  between  in-service  teachers  and  software  design  students  appeared  beneficial  based  on 
the  initial  needs  assessment,  precautionary  measures  were  implemented.  Before  production  began  on  the  software 
development  project,  it  was  deemed  necessary  to  assign  tasks  and  specify  roles.  This  was  an  attempt  to  prevent  the 
needs  of  cither  party  from  being  ignored,  which  could  potentially  foster  animosity  towards  future  collaboration  and 
could  hinder  the  completion  of  this  particular  project.  As  such,  a collaborative  model  that  takes  into  account  such 
problems  as  role  ambiguity  and  project  coordination  was  developed  specifically  for  this  type  of  project  (sec  Fig.  1 ). 
Descriptions  of  each  phase  of  the  model  are  detailed  in  the  following  section  along  with  narrative  describing  the 
outcomes  of  each  phase  for  this  particular  software  development  project. 

Description 

The  Software  Collaboration  Model  is  based  on  traditional  insiructional  design  principles  and  is  described  in 
terms  of  phases  (interest,  equipment,  etc.)  but  also  considers  the  participants  (software  students,  teacher).  For  each 
phase,  participants  have  specific  duties  to  perform  or  decisions  to  make.  Just  as  a good  instructional  design  model 
will  lead  to  good  instruction,  when  these  phases  are  adhered  to,  it  is  postulated  that  the  result  will  he  a final  product 
that  benefits  all  parties  involved.  However,  if  at  any  time  during  the  project  the  needs  of  one  of  the  parties  are  not 
being  met  and  a resolution  does  not  occur,  it  is  suggested  that  the  collaboration  be  terminated.  Without  an  end 
product  that  benefits  both  parties,  the  collaboration  does  not  serve  the  intended  purpose  and  the  use  of  the  model  is 
nothing  more  than  a formality. 

Interest 

The  first  phase  of  this  model  is  critical  to  the  successful  completion  of  a product.  The  first  phase  is  entitled 
“Interest”  and  involves  two  separate  tasks.  The  first  task  is  to  identify  the  roles  of  the  participants.  According  to 
traditional  instructional  design  principles,  it  is  crucial  to  identify  the  roles  of  those  participating  in  the  instructional 
design  process  (Dick  & Carey,  1996;  Kemp,  Morrison,  and  Ross,  1994).  Once  roles  are  determined  and  agreed 
upon,  there  is  little  to  no  ambiguity  as  to  the  purpose  of  each  participant  throughout  the  process. 
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Figure  1.  Software  Collaboration  Model 

For  this  particular  project,  an  introductory  meeting  was  held  to  assign  roles.  The  teacher  was  identified  as 
the  content  expert  and  the  software  design  students  as  the  programming  experts.  It  is  no  coincidence  that  the 
outcomes  of  this  task  mirrored  the  identification  of  subject-matter  experts  and  media  specialists,  tasks  identified  in 
the  instructional  design  models  created  by  Kemp,  Morrison,  and  Ross  (1994)  and  Dick  and  Carey  (1996). 

The  second  task  involved  in  the  “Interest”  phase  includes  the  identification  of  the  content  to  be  used. 
Because  this  is  a project  that  is  to  benefit  both  parties,  both  parties  must  be  interested  in  developing  the  same  content 
area.  This  step  is  very  similar  to  the  specification  of  content  step  in  the  Gerlach  & Ely  Model  (Gerlach  & Ely,  1980) 
and  the  identification  of  needs  and  goals  step  of  the  Dick  and  Carey  Model  (Dick  & Carey,  1996). 

Essentially,  the  purpose  of  this  task  is  to  identify  the  needs  of  both  parties  and  specify  the  content  area  that 
will  meet  those  needs.  The  content  area  that  was  specified  in  the  introductory  meeting  for  this  specific  project  was 
6th  grade  math.  This  content  area  was  particularly  important  to  the  teacher,  but  was  also  an  area  in  which  the 
software  students  felt  they  could  market  their  program. 

Equipment 

Because  the  instruction  being  considered  is  computer-based,  the  second  phase  of  this  collaborative  model 
involves  the  identification  of  and  access  to  sufficient  equipment.  Design  models  developed  for  computer-based 
instruction  by  Hannafin  and  Peck  (1988)  and  Tripp  and  Bichclmeycr  (1990)  assert  that  the  identification  of 
computer  resources  is  an  essential  aspect  to  the  design  process.  As  with  those  models,  this  particular  model  forces 
the  participants  to  identify  and  recognize  limitations  that  equipment  may  have  on  instructional  development. 

In  this  particular  project,  the  design  students  toured  the  school  and  determined  that  the  equipment  in  the 
school  was  modern  enough  to  run  the  type  of  program  that  they  needed  to  produce.  At  the  same  time,  the  teacher 
examined  the  computer  lab  schedule  and  determined  that  she  had  a sufficient  supply  of  computers  as  well  as 
adequate  access.  As  a side  note,  had  cither  of  the  participants  felt  that  their  needs  were  not  being  met  due  to 
equipment  constraints  or  varying  interests,  the  software  project  would  have  ceased  at  this  point. 
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Program  Design 

As  with  the  “Equipment”  phase,  the  “Program  Design’’  phase  needs  to  consider  the  limitations  of  the 
media.  However,  other  design  limitations  must  also  be  considered  at  this  point.  In  order  to  accomplish  this,  the 
subject-matter  expert  (teacher)  and  media  experts  (design  students)  must  collaborate  to  determine  the  feasibility  of 
several  design  options  given  the  chosen  content  area.  This  phase  is  similar  to  the  step  in  Dick  and  Carey’s  (1996) 
instructional  design  model  where  the  selection  of  instructional  materials  and  development  of  instructional  strategies 
occurs.  In  essence,  the  specific  focus  of  this  phase  is  the  mental  creation  of  what  the  instruction  will  look  like  when 
finished,  taking  into  consideration  equipment,  programming,  and  content  limitations. 

The  specifics  of  this  phase  for  this  particular  project  are  detailed  and  lengthy.  Therefore,  the  report  on  this 
section  is  somewhat  abbreviated.  In  this  particular  phase,  the  teacher  became  the  center  of  attention  as  she  supplied 
the  content  materials  and  expertise  in  the  content  area.  She  discussed  several  math  units  that  could  be  covered  and 
provided  presentation  ideas,  instructional  methods,  and  materials  to  the  software  design  students.  The  students, 
along  with  the  teacher,  discussed  the  ideas  while  taking  into  consideration  programming  abilities  and  time 
limitations.  In  the  end,  specific  units  were  determined  and  the  teacher  provided  books  and  resources  to  the  design 
students.  Presentation  and  instructional  methods  were  also  determined  for  each  unit.  Both  the  teacher  and  the 
design  students  brainstormed  potential  methods  and  when  they  were  finished,  everyone  had  a concrete  mental 
picture  of  what  the  instruction  would  look  like  when  complete. 

Development 

While  the  subject-matter  expert  (teacher)  is  the  center  of  attention  in  the  previous  phase,  the  media  experts 
(design  students)  are  the  center  of  attention  in  the  “Development”  phase.  During  this  period,  the  media  experts  arc 
charged  with  the  task  of  actually  creating  the  computer-based  instruction,  while  the  subject  matter  expert  (teacher) 
provides  content  advisement  and  any  additional  materials  that  are  needed.  This  phase  is  in  accord  with  the  design 
phase  described  in  the  Rapid  Prototyping  Model  (Tripp  & Bichelmeyer,  1990)  and  the  development  phase  of  the 
Hannafin  and  Peck  CAI  Model  (Hannafin  & Peck,  1988).  Ultimately,  the  outcome  of  this  phase  is  a prototype 
program  ready  for  testing. 

During  the  “Development”  phase  of  this  particular  project,  several  activities  occurred  that  hindered  the 
remaining  project  phases.  First  and  foremost,  the  media  experts  (students)  found  that  they  were  quite  the  optimists 
when  designing  the  program  and  had  not  been  realistic  in  some  of  their  design  decisions.  Because  of  the  linearity  of 
this  collaborative  model,  there  were  no  opportunities  to  go  back  to  previous  phases  and  revise.  Thus,  the  students 
felt  as  though  they  were  constantly  rushed  during  the  production  of  the  computer  software  and,  as  such,  did  not 
produce  the  level  of  quality  that  they  had  anticipated. 

The  second  barrier  that  appeared  during  this  phase  was  one  involving  communication.  Up  until  this  point, 
each  phase  was  addressed  in  face-to-face  meetings  between  the  teacher  and  the  design  students.  However,  once 
development  began,  the  meetings  ceased  to  occur  and  communication  was  limited  to  e-mail.  As  students  found  that 
they  needed  more  content  resources  than  was  initially  provided  by  the  teacher,  they  attempted  to  contact  the  teacher 
via  e-mail.  Because  the  teacher  was  not  a regular  user  of  e-mail  and  did  not  respond  immediately  to  their  requests, 
the  students  felt  as  though  their  progress  was  impeded  due  to  the  lack  of  communication  with  the  teacher. 

The  aforementioned  factors,  when  combined,  led  to  a feeling  of  non-cooperation  and  frustration  on  the  part 
of  the  media  experts  (students).  As  such,  the  students  decided  to  fore-go  the  use  of  the  subject-matter  expert 
(teacher)  and  continue  the  development  on  their  own.  Hence,  as  was  discussed  at  the  beginning  of  this  model,  one 
party  felt  that  their  needs  were  not  being  met  and,  accordingly,  the  project  as  whole  was  dissolved. 

Testing/Training 

The  “Testing/Training”  phase  of  the  Software  Collaboration  Model  involves  the  testing  of  the  program  and 
the  training  of  the  teachers.  This  closely  resembles  the  phase  in  the  Rapid  Prototyping  Model  (Tripp  & 
Bichelmeyer,  1990)  where  the  product  is  prototyped,  and  researched.  The  task  of  the  design  student  during  this 
phase  involves  the  training  of  faculty  who  plan  to  use  the  new  software.  The  design  students  are  also  responsible  for 
testing  the  program  on  selected  students  to  help  identify  potential  problems.  Those  students  are  chosen  by  the 
teacher,  who  is  also  responsible  for  receiving  training  on  the  new  software  at  this  time.  Essentially,  the  outcome  of 
this  phase  includes  a trained  teacher  who  can  utilize  the  program  and  a prototype  session  in  which  potential 
problems  arc  identified  and  fixed  before  the  actual  implementation.  (Due  to  the  fact  that  the  design  students  in  this 
particular  project  chose  to  dissolve  their  involvement,  specific  data  for  this  phase  is  not  available.) 


Implementation 

The  final  phase  of  the  collaboration  model  involves  the  mass  implementation  of  the  program  into  the 
curriculum.  The  task  of  curriculum  integration  rest  with  the  teacher,  as  does  the  scheduling  of  the  computer  lab.  At 
this  point,  the  primary  task  for  the  design  students  is  to  become  a provider  of  technical  support  and  assistance  when 
needed.  The  design  students  may  also  take  this  opportunity  to  gather  Beta  data  concerning  their  program.  This 
phase  most  closely  resembles  the  task  in  the  Rapid  Prototype  Model  (Tripp  & Bichelmcyer,  1990)  known  as  the 
“Install  and  Maintain  System”  step.  (Due  to  the  fact  that  the  design  students  in  this  particular  project  chose  to 
dissolve  their  involvement,  specific  data  for  this  phase  is  not  available.) 

Revisions  to  the  Model 

Based  on  the  results  of  the  initial  prototype  project  of  the  Software  Collaborative  Model,  specific  revisions 
have  been  recommended  and  implemented.  First  and  foremost,  the  need  for  continual  adaptation  and  revision  of 
each  phase  is  a necessity.  Because  the  previous  model  lacked  the  revision  process,  the  software  design  students  felt 
undue  pressure  to  adhere  to  content  decisions  that  were  made  during  the  design  phase  even  though  they  were  found 
to  be  impractical  during  the  development  phase.  To  adjust  and  allow  for  revisions,  the  model  has  been  updated  and 
now  includes  a revision  process  at  the  design,  development,  testing/training,  and  implementation  phases  (see  Fig.  2). 
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Figure  2.  Revised  Software  Collaboration  ../odd 

The  second  revision  to  the  Software  Collaboration  Model  involves  communication.  Rather  than  address 
methods  of  communication  in  the  initial  “Interest”  phase,  the  topic  was  overlooked  and  eventually  led  to  the 
disintegration  of  the  prototype  software  design  project.  To  prevent  such  an  instance  from  occurring  again,  a task  has 
been  added  to  the  “Interest”  phase  (sec  Fig.  2).  Initially,  the  two  tasks  of  the  interest  phase  included  agreeing  on 
content  and  roles.  The  revised  model  adds  the  task  of  predetermining  communication  methods  to  this  phase.  By 
predetermining  the  methods  to  be  used,  participants  utilizing  the  model  may  encounter  less  difficulties  with  the 
communication  process. 

The  third  and  final  revision  that  is  undergoing  consideration  involves  the  instructional  design  component  of 
the  model.  During  the  “Program  Design”  phase,  the  present  model  relies  on  the  knowledge  of  both  the  teacher  and 
the  software  design  students  to  ensure  that  appropriate  instructional  design  procedures  arc  followed.  However,  it  is 
proposed  that  the  involvement  of  a third  party  who  specializes  in  instructional  design  may  enhance  the  instructional 
quality  of  the  final  produced  program.  At  the  present,  the  current  author  is  experimenting  with  this  idea  by  adding  a 
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masters  level  instructional  technology  student  to  the  project.  The  benefits  for  both  the  instructional  technology 
student  and  the  final  software  program  are  currently  being  explored,  but  have  yet  to  be  added  to  the  model.  If 
successful,  this  aspect  will  be  added  to  the  Software  Collaboration  Model. 

Conclusion 

This  paper  has  briefly  described  the  needs  and  benefits  of  a collaboration  between  senior  software  design 
students  and  public  teachers.  Based  on  those  needs  and  benefits,  the  Software  Collaboration  Model,  which  mirrors 
aspects  of  traditional  instructional  models,  was  developed.  To  test  the  collaborative  model,  a group  of  software 
design  students  and  a teacher  were  chosen  to  participate  in  the  collaborative  development  of  a software  design 
project.  The  results  of  this  prototype  project  that  attempted  to  utilize  the  Software  Collaboration  Model  was 
reported.  From  those  results,  the  areas  of  communication,  revision,  and  instructional  design  were  identified  as 
potential  areas  in  need.  A revised  version  of  the  model  was  developed  and  described  that  took  into  account  two  of 
these  three  areas.  At  this  point,  the  call  is  for  future  prototyping  and,  if  necessary,  future  revision  to  the  model. 
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Abstract 

Contemporary  computer-based  environments  use  the  capabilities  of  technology  to  create  contexts  which 
require  and  afford  students  the  opportunity  to  engage  in  authentic  problem  solving  by  generating,  testing,  and 
revising  hypotheses;  exploring  and  manipulating  components  of  the  environment;  and  representing  and  reflecting 
on  what  they  know.  By  design , such  environments  require  sophisticated  levels  of  cognitive  functioning  including 
higher  order  cognitive  and  metacognitive  functioning.  Learning  in  open  learning  environments  relies  on  a shared, 
reciprocal,  and  co-constrticted  understanding  of  events  between  the  learners  and  the  learning  environment.  The 
purposes  of  this  paper  are  to  critically  analyze  problems  and  issues  related  to  learning  with  open,  technology-based 
environments. 

Open-ended,  learner-centered  environments  arc  designed  to  support  individual  intentions  to  construct 
personal  meaning  by  engaging  in  authentic  projects  or  solving  problems.  The  philosophical  and  theoretical 
assumptions  of  student-centered  learning  are  fundamentally  constructivist  and  situated  (Jonassen,  1991;  Hannafin  & 
Land,  1997).  Methods  for  supporting  personal  knowledge  construction,  such  as  problem-based  learning  (Savcry  & 
Duffy,  1996),  anchored  instruction  (Cognition  and  Technology  Group  at  Vanderbilt,  1992),  situated  cognition 
(Brown,  Collins,  & Duguid,  1989),  project-based  learning  (Blumcnfeld,  ct  al.  1991),  and  open  learning 
environments  (Hannafin,  Hall,  Land,  & Hill,  1994),  share  those  constructivist  and  situated  assumptions  about 
learning  and  aim  to  provide  contexts  and  structures  that  support  meaningful,  learner-centered  interactions. 

In  concert  with  these  methods,  technology  is  often  employed  as  a tool  or  mediator  of  the  process, 
functioning  as  cognitive  tools  for  experimentation,  manipulation,  and  generation  of  ideas.  The  result  is  a complex, 
reciprocal  interaction  of  learning  with  technology;  tools,  resources,  and  scaffolding  facilitate  actions  that  augment 
thinking,  and  meaning  is  built  upon,  and  governed  by,  the  results  of  these  (learner-driven)  actions  (Salomon, 
Perkins,  & Globerson,  1991.  Achieving  these  results,  however,  requires  continual  evolution  and  sharing  of 
meanings  within  the  learner-technology  partnership.  The  purposes  of  this  p per,  then,  are  to  clarify  (1)  the 
theoretical  assumptions  that  underlie  learning  with  open,  technology  environments;  (2)  limitations  in  adaptive 
capabilities  of  technology;  (3)  the  significance  of  two-way,  shared  understanding;  and  (4)  problems  and  issues  that 
can  arise  during  the  co-construction  process. 

Learning  with  Open  Learning  Environments:  Processes  and  Assumptions 

OLEs  are  technology-based  environments  that  require  learners  to  deploy  unique  knowledge  and  skill  to 
solve  complex  and  authentic  problems  (Hannafin,  Hall,  Land,  & Hill,  1994).  They  use  the  capabilities  of 
technology  to  create  contexts  wherein  complex  concepts  can  be  represented,  manipulated,  explored,  and  revised. 
Rich  information  resources  and  computer-based  tools  are  woven  into  contexts  that  create  virtual  spaces  to  support 
problem-solving  and  higher-order  thinking.  Examples  of  OLEs  include  computer-based  inicroworlds  (sec 
Microworlds  Project  Builder,  1993);  resource-based  hypermedia  environments  for  researching  and  linking  ideas 
such  as  cognitive  flexibility  hypertexts  (Spiro,  Fcltovich,  Jacobson,  & Coulson,  1991);  and  collaborative  knowledge 
construction  cnvircnmcnis  for  sharing  and  linking  of  evidence,  questions,  and  products  (Linn,  Bell,  & Hsi,  in  press). 
OLEs  foster  development  of  understanding  by  providing  opportunities  for  learners  to  identify,  test,  and  revise 
ongoing  theories  as  they  solve  complex  problems  and  integrate  multiple  perspectives  (Land  & Hannafin,  1996). 

The  centrality  of  the  learner  in  defining  meaning  is  considered  a necessary  requirement  for  combating 
problems  of  over-simplified  (Spiro,  et  al.,  1991),  “compliant, ” (McCaslin  & Good,  1992)  and  deeontextualized 
thinking  (Cognition  and  Technology  Group  at  Vanderbilt,  1992).  Understanding  is  assumed  to  be  best  supported 
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when  it  is  rooted  in  personal  experience  and  is  accomplished  via  exploration,  interpretation,  and  negotiation 
(Hannafin  & Land,  1997).  According  to  Land  & Hannafin,  (1996)  understanding  evolves  continuously  and 
dynamically,  as  ideas  are  generated,  tested,  and  revised. 

While  assumptions  and  theories  of  student-centered  learning  are  provocative,  many  questions  remain  about 
the  efficacy  of  such  environments,  specifically  those  that  are  technology-based.  For  example,  evidence  indicates 
that  learners  do  not  always  use  open  systems  in  ways  that  support  knowledge  construction,  so  they  fail  to  evolve 
conceptions  significantly,  and  may  not  use  the  affordcinces  of  technology  to  develop  meaningful  understanding 
(Atkins  & Blissetl,  1992).  Rather  than  engaging  in  reflective  learning,  learners  often  deploy  random  instead  of 
reason-based  actions,  search  for  “answers’*  instead  of  generating  and  testing  problems  (Wallace  & Kupperman 
(1997),  and  sustain  naive  beliefs  (in  the  face  of  repeated  conflicting  evidence)  instead  of  evolving  them  (Land  & 
Hannafin,  1997).  Perkins  (1985)  has  warned  against  the  notion  of  a “fingertip  effect”  — an  over-reliance  on 
assumptions  about  what  an  environment  can  afford  - in  that  the  existence  of  an  opportunity  does  not  imply  that  it 
will  be  shared  or  engaged. 

Explanations  for  the  lack  of  effectiveness  include  limitations  of  the  system  to  adapt  to  student  needs, 
unanticipated  implementation  requirements  of  teachers,  and  a lack  of  empirical  grounding  for  selected  approaches, 
methods,  or  desired  outcomes  (Hannafin,  Hannafin,  Land,  & Oliver,  1997).  Additionally,  a host  of  psychological 
and  motivational  variables  may  also  impact  understanding  in  open  systems.  In  order  for  meaningful  learning  to 
occur,  systems  rely  significantly  on  the  intentions,  actions,  processing,  and  regulation  mechanisms  of  the  learner. 
Regardless  of  contexts  or  content  domain,  learners  must  assume  primary  responsibility  for  the  learning  process. 
This  charge  requires  them  to  derive  goals  and  problems,  make  informed  decisions,  lake  reason-based  actions, 
process  information  and  feedback,  and  monitor  their  needs  and  approaches  on  a meta-level.  Furthermore,  in  order  to 
continually  evolve  conceptions,  learners  must  derive  interpretations,  test  them  for  their  validity,  and  evaluate  their 
utility  in  light  of  confirming  or  conflicting  data.  These  requirements  for  effectively  functioning  in  learner-centered 
systems  are  demanding  and  exacting  and  not  typically  possessed  by  students.  Yet,  assumptions  dictate  that  learners 
arc  not  only  capable  of  meeting  such  psychological  requirements,  but  they  arc  also  imminent  when  supported  by  the 
system  context. 

Co-Construction  with  Technology:  Issues  in  Adaptability 

A fundamental  premise  of  constructivist  learning  is  that  learners  evolve  understanding  as  a direct  result  of 
encountering  “discrepant  events”,  failed  goals,  unmet  expectations,  problem",  curiosities,  or  cognitive  dissonance 
(Piaget,  1976).  This  implies  that  learners  naturally  develop  understanding  of  the  surrounding  environment  by 
attempting  to  make  sense  of  it  and  reconcile  deviations  from  expectations.  Yet,  clearly  there  are  also  instances 
where  “natural  teaching”  — a reactive  response  to  student  needs  when  understanding  is  arrested  — is  also  effective 
for  guiding  students  to  a more  refined  conceptual  level  (Schank  & Cleave,  1995).  The  zone  of  proximal 
development  implies  that' meaningful  learning  occurs  as  a result  of  learner  engagement  in  activities  that  are 
“scaffolded”  by  social,  material,  or  technological  faculties  (Vygotsky,  1978).  Thus,  the  development  potential  of 
learners  is  extended  with  the  help  of  dynamic  support  that  assists  them  in  building  upon  initial  understanding. 
However,  the  learner’s  ability  to  perceive  and  act  upon  a “need  to  know,”  as  well  as  the  environment’s  ability  to 
respond  to  these  needs,  arc  critical  to  realizing  effective  interaction  within  the  zone  of  proximal  development. 

Yet,  re-directing,  responding  to,  or  meeting  learner- gene  rated  needs  within  the  theoretical  “zone”  is  often 
problematic  for  technology-enhanced  environments.  One  reason  for  this  difficulty  is  that  explicit,  a priori 
identification  of  expected  student  needs  arc  difficult  to  predict  and  thus  fulfill.  Whereas  general  scaffolding  can  he 
provided  (e.g.,  help  resources,  instructional  support,  guiding  questions,  hints,  or  prompts  for  reflection),  truly 
dynamic  or  “intelligent”  responses  to  unique  issues,  interpretations,  or  questions  may  not  be  possible  to  satisfy 
through  technology  support  alone.  Potential  difficulties  are  exacerbated  when  learners  deploy  idiosyncratic 
preferences  and  approaches  that  are  not  supported  (nor  sometimes  desired)  by  the  design  (e.g.,  searching  for  a right 
answer  vs.  u:  ng  tools  to  experiment).  When  learners'  goals  arc  dissonant  from  system  goals,  interactions  become 
distorted,  and  the  environmental  support  is  counterproductive. 

Limitations  in  adaptive  capabilities  of  OLEs  are  significant  for  several  reasons.  First,  the  building  and 
evolving  of  learner  understanding  is  contingent  upon  feedback  that  is  correspondingly  contingent  upon  learner 
actions.  The  environment  affords  cognitive  performance  only  when  learner  intentions  and  perceptions  arc  consonant 
with  system  goals.  With  microworlds  and  simulations,  for  instance,  learners  must  construct  a “working  model”  of 
their  understanding  that  is  progressively  honed  via  manipulation  of  system  features.  Through  interaction,  learners 
make  their  ideas  explicit  and  clarify  and  extend  both  their  understanding  and  implicit  models  (Driver  & Scanlon, 
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1988).  Feedback  about  the  efficacy  of  the  model  is  essential  for  critical  examination,  reflection,  and  conceptual 
development.  As  intentions  and  actions  are  increasingly  linked  with  the  feedback  (and  subsequent  processing  of  the 
feedback),  understanding  is  generated  and  examined  critically. 

Second,  the  importance  of  adaptive  feedback  becomes  even  more  critical  when  learners  maintain  naive 
beliefs  and  fragmented  understanding  that  are  extremely  resilient  to  change  (Chinn  & Brewer,  1993).  Furthermore, 
they  arc  expected  to  hold  naive  theories  that  will  be  represented  in  their  actions  and  in  their  intentions  to  test  their 
validity.  Yet,  evolving  enduring  beliefs  requires  repeated  exposure  to  counter-examples  that  indicate  how  existing 
beliefs  are  limited  in  explanatory  power  (Karmiloff-Smith  & Inhelder,  1975;  Piaget,  1976;  Vosniadou,  1992). 
Schon  (1983)  notes:  “Doing  extends  thinking  in  the  tests  ...  of  experimental  action,  and  reflection  feeds  on  doing  and 
its  rcsuks...It  is  the  surprising  result  of  action  that  triggers  reflection...”  (p.  280).  Thus,  the  system  must  be  able  to 
“understand”  formative  thcorics-in-action,  represent  the  consequences  or  effectiveness  of  them,  and  provide 
responsive  data  that  can  be  perceived  as  consistent  or  inconsistent. 

Yet,  even  when  the  system  provides  responsive  feedback  about  learners'  models,  if  they  fail  to  relate  that 
feedback  to  their  theories,  they  may  not  perceive  how  data  support  or  contradict  their  beliefs.  Intuitive  theories  (c.g., 
impetus  theories  of  force  and  motion  [Driver  & Scanlon,  1988],  beliefs  that  the  Earth  is  Hat,  and  beliefs  that  density 
is  not  a factor  in  the  buoyancy  of  an  object)  arc  so  consistently  reinforced  through  personal  experience  that  they 
form  the  foundation  of  firmly-established,  though  often  fundamentally  flawed,  theories.  In  some  cases,  underlying 
beliefs  arc  so  entrenched  that  learners  fail  to  consider  testing  them.  If  one  truly  believes  that  heavier  objects  sink  and 
lighter  objects  float,  for  instance,  it  is  unlikely  that  one  will  intentionally  seek  out  opportunities  to  prove  it  wrong. 

It  is  unlikely  that  problems  in  meeting  the  requirements  or  assumptions  of  student-centered  learning  happen 
unilaterally,  i.c.,  that  the  locus  of  the  problem  lies  solely  within  the  learner  (i.c.,  he  or  she  is  incapable  of 
representing  meaning  or  accurately  perceiving  feedback)  or  solely  within  the  system  (feedback  is  incapable  of 
accurately  modeling  learner  understanding  or  responding  to  individual  needs).  Rather,  it  is  more  likely  that 
breakdowns  occur  as  a result  of  distorted  interactions  that  occur.  Thus,  the  success  of  a constructivist  environment 
hinges  upon  a shared,  reciprocal,  and  co-constructed  understanding  of  events  and  intended  meanings  during  critical 
or  teachable  moments.  That  is,  in  order  to  build  upon  understanding  in  meaningful  ways,  perceptions,  intentions, 
approaches,  interpretations,  questions,  and  feedback  must  be  understood  or  shared  among  participants.  This 
assertion  defines  critical  components  of  the  process  of  progressive  negotiation  and  differentiation  of  meaning: 
learners  must  generate  understanding  that  is  based  upon  existing  beliefs  as  well  as  responsive  feedback  from  the 
system  (and/or  others);  the  system  must  then  represent  or  model  the  intended  interpretation  and  provide  feedback 
regarding  its  efficacy;  the  learner  must  then  perceive  the  intended  meaning  of  data  or  feedback  and  refine 
interpretations.  The  cycle  continues  until  learners  identify  beliefs,  test  them  for  their  validity,  and  evolve  them  based 
on  responsive  feedback  and  scaffolding  from  the  system.  Thus,  in  order  to  support  reciprocity  and  sharing  of 
meaning,  both  models  must  be  adaptable  or  are  able  co-construct.  In  this  sense,  meaning  is  based  upon  a shored 
overlap  of  understanding  between  the  learner  and  the  system. 

The  remaining  sections  detail  issues  related  to  the  notion  of  shared  meaning  in  constructivist,  technology- 
based  systems  and  identify  potential  sources  of  misunderstandings. 

The  Importance  of  Shared  Meaning 

Shared  meaning  assists  both  the  learner  and  the  system  (or  instructor  or  other  learners)  to  increase  the 
accuracy  of,  or  decrease  the  potential  distortion  of,  the  co-construction  process.  Most  teachers  know  all-to-well  how 
easy  it  is  for  misunderstandings  to  take  place.  It  js  not  uncommon  to  hear  statements  from  learners  such  as  “I 
thought  you  meant ...”  or  “I  didn’t  realize  what  you  meant  by  ...”  or  “I  assumed  that  this  meant...”  when  recognizing 
that  misunderstandings  have  occurred.  Upon  receiving  such  feedback,  most  teachers  can  adapt  their  teaching  or 
communication  methods  to  clarify  the  meaning,  redirect  the  student’s  understanding,  or  to  circumvent  similar 
misunderstandings  from  taking  place  in  the  future.  Engaging  in  two-way  conversati  »ns  allows  both  participants  to 
cheek  to  sec  if  they  arc  understanding  each  other  and  recast  or  revise  explanations  accordingly. 

Shared  meaning  with  OLEs  implies  a functional  rclatedness  between  a set  of  meanings  and  messages 
shared  by  both  the  learner  and  system.  This  does  not  imply  that  meanings  arc  communicated  isomorphically;  rather, 
that  important  components  of  one  message  arc  received  and  used  as  the  basis  for  response  by  the  other.  There  is  no 
“one  way”  to  respond  or  make  sense  of  a meaning.  There  is  simply  a congruence  among  all  messages.  In  essence, 
shared  meaning  is  process  that  ensures  that  the  system  and  the  learner  are  operating  on  the  same  “page”  or  on  the 
same  collection  of  meanings. 


In  action,  the  concept  of  shared  meaning  involves  a dialectic  or  dynamic  communication  of  meanings. 
Messages  are  continually  sent  back  and  forth,  as  learners  perceive  important  variables,  generate  hypotheses,  and  test 
for  their  veracity.  When  learners  communicate  messages  to  the  system  through  the  interface,  the  system  is 
programmed  to  respond  to  them.  For  instance,  if  a learner  is  exploring  the  effect  of  interest  rate  on  monthly  housing 
payments  and  increases  the  interest  rate,  the  system  processes  it  and  returns  a message  that  is  based  upon  the  initial 
input.  That  is,  the  system  sends  a message  about  the  change  in  calculated  output  based  upon  the  learner’s  request  to 
increase  the  interest  rate.  The  learner  then  uses  this  message  as  the  basis  for  revising  an  interpretation,  generating  a 
new  hypothesis,  or  confirming  an  expectation.  The  system  response  should  always  be  contingent  upon  the  learner’s 
original  meaning  and  modeled  to  the  learner.  Then,  a collection  of  new  meanings  or  responses  are  generated  by  the 
learner,  yet  they  must  be  built  upon  original  meanings  and  also  accommodate  the  system’s  message.  The  cycle  is 
repeated  over  time,  with  every  message  being  used  to  refine  a preceding  one,  until  the  learner  decides  to  end  the 
interaction. 

The  shared  meaning  process  is  significant  in  that  it  lies  at  the  root  of  many  assumptions  about  how 
understanding  evolves  with  OLEs.  It  is  assumed,  for  instance,  that  the  learner  can  represent  and  communicate  a 
meaning  to  the  system.  It  is  further  assumed  that  the  system  can  interpret  the  meaning,  model  it,  and  represent  a 
response  that  can  be  discerned  by  the  learner.  Finally,  it  is  assumed  that  the  learner  will  accurately  perceive, 
understand,  and  appropriate  the  message  sent  by  the  system,  without  altering  its  intention  or  ignoring  it  altogether. 
These  are  high  expectations,  particularly  when  considered  in  light  of  the  fact  that  a machine  cannot  understand  a 
message  that  it  has  not  been  programmed  a priori  to  process. 

Rather  than  judging  the  ’'correctness”  of  learners'  responses,  the  system  seeks  convergence  of  system  and 
learner  models.  Initially,  learner  models  may  be  quite  distinct  from  how  they  ultimately  develop.  Yet,  through 
interaction  with  the  system,  initial  and  desired  models  become  more  aligned  with  each  other  within  an  acceptable 
range  of  confidence  as  defined  by  a community  of  practice. 

As  such,  shared  meaning  should  not  be  considered  as  a goal  in  and  of  itself.  A tape  recorder,  for  instance 
can  accurately  model  or  report  exactly  what  has  been  dictated;  yet  a tape  recorder  does  not  understand.  Rather, 
shared  meaning  is  a necessary  and  important  means  toward  increasing  both  partners  understanding  of  each  other's 
point  of  view  at  important  points  during  the  interaction.  It  is  intended  to  emphasize  theoretically  the  need  for  both 
the  learner  and  the  system  to  occupy  a mutual  and  as  complete  understanding  as  possible  of  the  other’s  intended 
meanings.  This  does  not  imply  that  complete  understanding  of  all  of  the  possible  meanings  of  the  system  nor  all  of 
the  possible  meanings  of  the  learner  are  cither  possible  or  desired.  But,  it  is  assumed  likely  that  en-routc  interactions 
between  the  system  and  the  learner  will  result  in  a progressive  negotiation  and  development  of  understanding. 
Shared  meaning  is  important  for  open  systems  because,  when  it  occurs,  interpretations  can  move  forward  from  a 
point  of  understanding  versus  from  a point  of  misunderstanding. 

Sources  of  Misunderstanding 

When  the  shared  meaning  process  breaks  down  during  open-ended  learning,  distortions  in  approaches, 
perceptions,  or  interpretations  result.  The  purpose  of  this  section  is  to  illustrate  where  and  how  such  points  of 
departure  can  occur.  While  it  is  likely  challenging  or  unnecessary  to  foster  complete  sharing  of  meaning  for  all 
possible  learner-system  interactions,  specific  types  of  interactions  at  critical  points  in  the  process  can  seriously 
impact  the  extent  to  which  meanings  evolve. 

Perceptual  Limitations  of  Visual  Cues 

Meaning  is  shared  in  instances  where  learners  connect  system  data  with  personal  meaning,  and 
subsequently  expand  ways  of  explaining  phenomena.  As  a result  of  the  exchange,  informal  understanding  becomes 
more  clearly  connected  to  the  experiences  that  informed  them  and  to  formalized  concepts.  Accordingly,  system  and 
learner  models  are  continually  refined  and  merged.  Yet,  us  messages  and  intended  meanings  are  exchanged,  two 
processes  arc  essential  for  them  to  be  shared.  First,  intended  meanings  must  be  accurately  perceived  by  the  learner 
and/or  the  system.  Second,  observed  data  must  be  accurately  appropriated  with  interpretations  that  arc  constructed. 
For  instance,  if  a learner  virtually  manipulates  the  effect  of  mass  on  an  object’s  acceleration,  and  the  system 
provides  feedback  indicating  Miat  the  increase  in  mass  did  not  increase  its  acceleration,  it  is  essential  that  the  learner 
is  able  to  perceive  that  acceleiution  has  not  increased.  Furthermore,  if  the  learner  concludes  from  that  feedback  that 
two  objects  of  different  masses  will  fall  to  the  Earth  at  the  same  speed,  he  or  she  has  constructed  a meaning  that  is 
consonant  with  the  message  sent  by  the  system  and  is  linked  to  empirically-based  observations.  Breakdowns  in 
either  the  accurate  perception  or  appropriation  of  system  messages  can  result  in  distorted  conceptions.  In  order  for 
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understanding  to  evolve  based  on  shared  meanings,  learners  must  induce  personal  models  to  unify  multiple  system 
concepts,  accurately  connecting  factors  and  influences  into  wholes  versus  parts. 

The  ability  to  draw  accurate  conclusions  based  on  empirical  data  is  integral  to  scientific  inquiry  (Roth  & 
Roychoudhury,  1993).  Similarly,  during  the  shared  meaning  process,  learners  must  also  slay  proximal  to  the  data  - 
i.c.,  system  feedback  is  accurately  perceived  and  acted  upon  and  related  meanings  arc  used  to  annotate  and 
understand  the  data.  Drawing  conclusions  based  on  accurate  perceptions  is  critical  because  understanding  is  initially 
built  as  learners  attempt  to  make  judgments  regarding  cause  and  effect  (c.g.,  “when  I decreased  mass,  there  was  no 
change  in  acceleration”);  Understanding  evolves  as  learners  perceive  information  inconsistent  with  a theory  (c.g.,  “I 
expected  that  the  heavier  object  would  fall  faster  but  it  did  not”),  provide  an  explanation,  and  collect  data  to  confirm 
or  refute  it.  Thus,  the  process  of  both  building  and  evolving  meaning  is  contingent  upon  accurate  perception  and 
connection  of  data  and  events  occurring  in  the  system  (accuracy  is  defined  as  the  intended  meaning  of  the  message 
sent  by  the  system). 

However,  limitations  in  perceptual  processes  can  lead  to  problems  in  establishing  cause-effect 
relationships  or  to  the  development  of  conceptions  based  upon  faulty  presumptions.  Richer  (1995),  for  instance, 
details  a story  of  Pcrcival  Lowell,  a prominent  astronomer  at  the  turn  of  the  century,  who  serves  as  an  example  of 
how  inaccurate  visual  perceptions  can  lead  to  erroneous  conclusions.  Using  the  telescopic  technology  available  at 
the  lime,  Lowell  reported  the  spotting  of  long  crossing  lines  on  the  Martian  landscape.  Lowell  concluded  from  these 
observations  that  the  lines  were  the  remnants  of  canals  constructed  by  an  ancient  civilization.  Unfortunately,  his 
perceptions  (and  consequently  interpretations)  were  inaccurate,  in  that  the  canals  turned  out  to  be  optical  illusions. 

This  same  type  of  analysis  can  also  be  applied  to  learners  using  visualization  tools  in  open  environments, 
due  to  the  use  of  visually-based  feedback  that  is  often  presented  with  ambiguous  meaning  or  fidelity.  OLEs  often 
use  dynamic  displays  of  numerical  data  or  animation  to  represent  visually  the  effects  of  a learner’s  actions,  rather 
than  directly  tell  the  meaning  and  interpretation  of  events.  Many  simulations  or  visualization  tools  such  as  Space 
Shuttle  Commander . ErgoMotion , Geometer's  Sketchpad,  and  Interactive  Physics  model  the  effects  of  learner  input 
or  designs  via  animated  and/or  video  simulations  (Richer,  1992;  1995),  The  visually-simulated  outcomes  provide 
data  regarding  the  extent  to  which  learner  designs  functioned  and/or  changed.  From  this  feedback,  learners  refine 
conjectures  and  hypotheses  and  confirm  or  refute  expectations. 

To  illustrate,  a learner  attempting  to  maneuver  a spacecraft  in  Space  Shuttle  Commander  can  sec  the  result 
of  applying  a lighter  mass  to  the  spacecraft  by  watching  it  move  across  the  screen  and  comparing  it  to  the  animated 
results  of  applying  a heavier  mass.  Yet,  following  this  animated  feedback,  the  learner  might  inaccurately  perceive 
that  the  spacecraft  moved  faster  when  he  or  she  applied  a lighter  mass.  The  fact  that  the  object  may  not  have 
increased  in  speed  contingent  upon  mass  is  inconsequential  to  the  fact  that  the  learner  perceived  that  it  did  in  fact 
accelerate.  In  this  instance,  the  message  conveyed  by  the  system  (i.c.,  mass  did  not  impact  acceleration)  was  not 
shared  by  the  learner;  rather  it  was  confounded  with  inaccurate  perceptions  of  visual  information  (i.c.,  it  went  faster 
when  I decreased  mass).  From  this  initial  misperception,  it  is  not  implausible  to  expect  that  distorted  conclusions 
would  likely  follow. 

Utilizing  less  visual  forms  of  feedback,  such  as  illustrating  changes  in  numerical  values,  may  not  alleviate 
the  tendencies  of  learners  to  rely  heavily  on  visual  cues  as  the  basis  for  information.  To  illustrate,  numerical  data  arc 
often  provided  to  learners  in  concert  with  animated,  simulated  feedback.  Yet,  Land  and  Hannafin  (1997)  reported 
that  learners  heavily  relied  upon  video  simulations  of  a roller  coaster  to  judge  relative  differences  in  its  speed,  and 
failed  to  seek  numerical  data  to  confirm  the  speed,  even  when  they  were  available  in  the  form  of  data  points.  In  this 
example,  judging  prec  ise  differences  in  speed  was  not  possible  using  visual  data  alone.  Thus,  most  errors  in 
interpretation  were  due  to  inaccurate  perceptions  of  visual  data  from  the  system,  without  deeper  processing  of  the 
numerical  data  that  could  have  been  used  to  confirm  or  challenge  meaning. 

A reliance  on  literal  interpretation  of  visual  cues  is  likely  due  to  novice  learner’s  focus  on  superficial  or 
surface  features  of  a problem  (Chi,  Glaser,  & Rees,  1982).  Novice  learners  tend  to  confuse  visibility  with  relevance 
and  apply  little  consideration  to  the  underlying  logic  of  their  selection  (Petre,  1995).  Thus,  perceptual  limitations  of 
learners  can  affect  the  “pragmatic  precision”  (Hawkins  & Pea,  1987,  p.  296)  or  degree  of  exactitude  sufficient  for 
explanation.  But,  while  novices  may  not  require  precise  sources  of  evidence  in  order  to  establish  meaning,  they 
nonetheless  use  what  they  perceive.  This  phenomena  is  known  as  top-down  processing  - “initial  information 
triggers  an  early  interpretation  against  which  all  subsequent  information  is  judged”  (Richer,  1995,  p.  53).  In 
instances  where  learner  perceptions  conflict  with  meanings  intended  to  be  conveyed,  misleading  attributions  of 
cause  and  effect  result  that  become  difficult  to  further  lest,  alter,  or  refine.  Misunderstandings  can  further  endure  in 
the  likely  event  that  the  system  is  unable  to  detect  that  faulty  perceptions  have  been  constructed. 


Biased  and  Confounded  Meanings 

Distorted  conceptions  are  not  solely  apparent  as  a consequence  of  misperccived  visual  data.  Learners  arc 
also  susceptible  to  the  effects  of  biased,  preconceived,  or  deeply  entrenched  intuitive  beliefs  that  can  influence  the 
precision  with  which  messages  arc  perceived  or  the  likelihood  of  existing  beliefs  to  change  (Driver  &.  Scanlon, 
1988).  Meanings  are  filtered  through  existing  beliefs  and  experiences,  and  they  become  distorted  when  learners  add 
to,  ignore,  or  change  system  messages  to  fit  existing  conceptions  (without  evaluation).  When  this  phenomena 
OLCurs,  learners  perform  the  cognitive  equivalent  of  “seeing  what  they  want  to  sec.”  Thus,  the  issue  for  open 
learning  environments  is  not  that  learners  hold  strong  beliefs  that  arc  at  odds  with  those  represented  in  the 
environment;  on  the  contrary,  novice  learners  are  expected  to  hold  naive  beliefs.  Rather,  problems  occur  to  the 
extent  that  messages  arc  altered  or  distorted  to  fit  into  existing  belief  systems,  without  evaluation  of  whether  the  data 
provides  evidence  to  the  contrary. 

Wilson  and  Brekke  (1994)  refer  to  this  phenomena  as  “mental  contamination”  and  define  it  as  “the  process 
whereby  a person  has  an  unwanted  judgment,  emotion,  or  behavior  because  of  mental  processing  that  is  unconscious 
or  uncontrollable.1’  (p.  1 17).  This  psychological  mechanism  rcsuUs  in  automatic  attributions  of  meanings  that  arc 
useful  during  new  learning  --  it  helps  learners  to  quickly  make  order  out  of  disorder.  Yet,  in  order  for  naive  beliefs  to 
evolve  in  open  systems,  learners  must  recognize  conflicting  data  and  eventually  evaluate  biases  or  sho*tcomings  in 
thinking.  Perception  of  dissonance  and  “meta-conccptual  awareness'*  (Vosniadou,  1992)  are  necessary  triggering 
mechanisms  forevaluation  to  occur. 

Research  on  the  effects  of  preexisting  beliefs,  however,  indicate  that  they  are  remarkably  pcrscvcrant  and 
resistant  to  change  (Champagne  et  al.,  1985;  Kardash  & Scholes,  1995;  Perkins  & Simmons,  1988).  Social  theories, 
such  as  beliefs  about  AIDS,  capital  punishment,  or  prejudice,  are  enduring  and  often  difficult  to  alter,  even  in  the 
face  of  examples  that  counter  beliefs  (Kardash  & Scholes,  1995).  Wilson  and  Brekke  (1994)  illustrate  the  unwanted 
impact  of  existing  social  beliefs  on  actions  in  the  following  excerpt: 

Professor  Jones  is  grading  papers  from  a small  seminar.  When  grading  Hernandez’s  paper,  undesirable 
mental  processes  are  triggered  by  the  fact  that  Jones  dislikes  Hernandez  and  knows  that  Hernandez  is  a 
member  of  a minority  group.  That  is,  Jonc’s  dislike  and  prejudice  taints  her  evaluation,  such  that  the 
evaluation  is  more  negative  than  it  would  otherwise  be.  Furthermore,  assume  that  Jones  would  agree  that 
this  is  unfair  and  would  prefer  not  to  be  influenced  by  her  prejudice.  This  example,  then,  fits  our  definition 
of  mental  contamination  in  that  Jone’s  judgment  was  influenced  by  unwanted  agents  (her  prejudice),  (p. 
119). 

In  this  example,  the  data  (the  quality  of  a student’s  paper)  arc  evaluated  in  light  of  existing  beliefs  that  arc 
undesirable  and  deeply  rooted  in  experience.  A disconnect  exists  between  the  accuracy  of  the  message  sent  by  the 
student,  and  the  way  it  was  interpreted  and  acted  upon  by  the  instructor.  In  order  for  the  biased  theory  to  be 
weakened  and  the  data  to  be  evaluated  accurately,  a competing  theory  must  be  strengthened  as  the  legitimacy  of  the 
original  theory  is  questioned  (Holland,  Holyoak,  Nisbctt,  & Thargard,  1986). 

Yet,  novice  learners  (particularly  young  learners)  do  not  spontaneously  test  theories  to  determine  if  their 
thinking  is  biased  (Karmiloff-Smith  & Inhclder,  1975;  Land  & Hannafin,  1997).  Rather,  they  require  repealed 
exposure  to  counter-examples  before  gradually  recognizing  limitations  in  thinking.  But,  even  when  conflicting  data 
about  the  usefulness  of  beliefs  are  repeatedly  provided,  initial  theories  may  remain  largely  unchanged.  Research 
examining  the  evolution  of  learner  beliefs  with  open  learning  environments  indicates  that  learners  often  fail  to 
perceive  and  interpret  data  as  being  inconsistent  with  existing  theories  (Land  & Hannafin,  1997).  Even  when 
learners  were  confronted  with  obvious  conflicting  evidence,  they  failed  to  act  on  the  inconsistencies  Instead,  they 
cither  changed  theories  temporarily,  without  acknowledging  the  previous  theory,  or  discounted  the  data  as  an 
exception  rather  than  attempting  to  explain  it.  Paradoxically,  when  confronted  with  counter-examples  based  upon 
data  that  is  ambiguous  (e.g,  imprecise  visual  data),  learners  often  manipulate  the  data  to  actually  strengthen  or 
confirm  the  existing  theory. 

Failure  to  Share  Meanings  within  the  System  Boundaries:  The  Situated  Learning  Paradox 

Another  assumption  underlying  the  design  of  OLEs  is  that  learning  is  optimized  when  it  is  situated  in  an 
authentic  context  (Hannafin  ct  al.,  1994).  Learning  is  facilitated  when  opportunities  to  connect  everyday  knowledge 
to  new  contexts  arc  provided.  As  learners  connect  system  experiences  with  prior  knowledge,  they  access  existing 


frameworks  lhal  strengthen  and  elaborate  new  understanding.  It  i widely  recognized  that  integrating  new 
knowledge  with  personal  experiences  results  in  more  meaningful  learning  (Mayer,  1984;  1989). 

When  learners  access  and  use  related  experiences  to  construct  and  annotate  meaning,  they  inherently 
extend  thinking  beyond  the  boundaries  delineated  by  the  system.  That  is,  they  access  experiences  that  arc  not 
represented  in,  nor  are  they  often  able  to  be  represented  in,  the  specific  environmental  context.  This  means  that 
meanings  accessed  from  personal  experiences  will  not  be  communicated  to,  nor  modeled  bv,  the  system.  As  staled 
previously,  meanings  are  shared  as  they  are  represented  and  communicated  via  available  interface  input/output 
options.  In  order  for  meanings  to  be  reciprocally  shared,  the  system’s  response  to  a learner’s  meaning  must  embody 
the  intended  model  or  message  as  well  as  additional  information  about  its  veracity.  When  learner  meanings  arc 
rooted  in  prior  experiences  and  cannot  be  tested  within  the  system  boundaries,  they  are  unable  to  be  accommodated 
or  shared  by  the  system.  While  this  is  not  inevitably  problematic,  it  nonetheless  illustrates  limited  capabilities  of  the 
system  to  adapt  to  and  model  learner  meanings  that  are  rooted  in  prior  experiences. 

To  the  extent  that  functionally  related  meanings  are  accessed  from  prior  knowledge,  problems  are  less 
significant.  Take  for  instance  a hypothetical  learning  environment  where  learners  can  explore  the  concept  of 
buoyancy  and  water  displacement.  The  interface  might  revolve  around  a virtual  swimming  pool,  where  learners  can 
observe  how  objects  of  various  weights,  sizes,  and  densities  float  or  sink.  The  leaner  may  observe,  for  instance,  that 
a small,  highly  dense  object  can  sink  while  a large,  less  dense  object  of  the  same  mass  does  not.  Based  on  this 
observation,  a learner  might  make  the  following  connection  to  prior  experience:  “This  sounds  like  what  happens 
when  I curl  up  into  a ball  in  a pool:  I sink.  But,  when  I extend  my  arms  and  legs  on  the  surface,  I float.  But,  I have 
the  same  weight.”  This  connection  represents  a functional  rclatedness  to  the  concept  represented:  The  meaning 
constructed  by  the  learner  contains  both  the  “message”  that  the  system  intends  to  send  (i.e.,  density  is  a more 
important  factor  than  weight  or  size  in  buoyancy)  as  well  as  other  related  meanings  based  on  prior  experiences  (c.g., 
“I  sink  when  I curl  into  a ball  vs.  when  I am  extended.”  These  meanings  are  functionally  congruent  and,  in  this  one 
interaction,  could  be  considered  shared.  While,  it  still  cannot  be  directly  tested  in  or  communicated  to  the  system 
(unless  system  can  spontaneously  change  its  interface  to  include  options  for  manipulating  the  weight  of  a child  in  a 
curled  vs.  extended  position),  the  result  is  a richer  conceptualization  that  is  now  strongly  rooted  in  experience. 

Yet,  while  links  to  prior  knowledge  enhance  the  potential  for  transfer  (Brown,  Collins,  & Duguid,  1989), 
they  also  increase  the  likelihood  that  learners  will  draw  upon  incomplete  and  often  inaccurate  understanding  which 
form  the  basis  of  faulty  theories.  To  illustrate  using  the  same  buoyancy  example,  a learner  could  also  conceivably 
make  the  following  connection  to  personal  experience:  “When  I wear  flotation  devices  on  my  arms  in  a pool,  I 
float.  When  I don’t,  I’ll  sink.  I don’t  sink  with  the  floats  because  they  make  me  lighter.”  Incomplete  conceptions, 
such  as  this,  are  typical  in  content  areas  like  science  where  everyday  experiences  arc  at  often  odds  with  scientific 
conceptions  (Carey,  1 986). 

Most  learners  think  that  heavier  objects  fall  to  the  Earth  faster  and  lighter  objects  float.  They  also  have  a 
collection  of  personal  experiences  to  support  these  intuitive  theories.  The  result  is  a set  of  robust  conceptions  that 
are  rooted  in  personal  experience  and  highly  resistant  to  change.  When  naive,  personal  theories  are  used  to 
inaccurately  annotate  the  meanings  communicated  by  the  system,  the  system  is  unable  to  detect  or  adapt  to  the 
conception.  Thus,  limitations  in  adaptive  capabilities  of  the  system  arc  significant  in  instances  where  inaccurate  or 
irrelevant  experiences  arc  accessed  and  used  to  explain  data  that  is  inconsistent  with  system  experiences  and  cannot 
be  tested. 

Similarly,  Land  and  Hannafin  (1997)  noted  that  learners  also  referenced  prior  knowledge  and  experiences 
in  the  ErgoMotion  roller  coaster  environment  that  contradicted,  or  interfered  with,  the  system’s  treatment  of  the 
concepts  of  force  and  motion.  One  learner  referred  to  the  regulating  effects  of  a computer  on  the  speed  of  the 
coaster,  and  the  use  of  brakes  and  “clamps”  for  stopping  the  coaster.  She  recalled  a roller  coaster  operator  telling 
her  that  the  coaster  had  brakes  and  could  stop  mid-ride.  Consequently,  she  used  this  information  to  interpret 
feedback  and  drive  future  actions.  She  continued  to  make  references  to  brakes  and  clamps  during  remaining  use  of 
the  system  when  addressing  issues  of  slowing  down  and  stopping.  Her  conceptions  did  not  evolve,  since  they  were 
strongly  rooted  in  experience  and  could  not  be  tested  using  the  available  tools.  Consequently,  she  never  confronted 
evidence  to  provoke  her  into  questioning  these  theories.  In  this  case,  a belief  that  the  coaster  slowed  down  around 
the  curve  (because  of  brakes  or  a computer  regulating  it)  interfered  with  an  understanding  of  force  and  motion  in  a 
context  that  did  not  support  exploration  of  the  notion. 

Such  interactions  can  lead  to  powerful  generalizations  since  they  are  linked  to  prior  knowledge  but  cannot 
be  easily  refined  using  system  tools  and  resources.  The  effect  is  further  intensified  in  that  prior  knowledge  and 
experience  are  exceptionally  robust  and  persistent.  Tlius,  when  learners  make  connections  to  prior  knowledge,  and 
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when  they  are  ill-informed  or  inaccurate,  they  cannot  be  tested  within  the  parameters  of  the  system  and  thus  may 
endure.  While  the  benefits  of  building  upon  meaningful  experiences  are  clear,  the  challenges  of  the  system  to  detect 
and  accommodate  them  remain  formidable. 

Failure  to  Share  the  Rules  of  Interaction:  Incongruent  Meta-Level  Approaches 

OLEs  assume  that  learners  are  responsible  for  their  own  learning  and  are  capable  of  monitoring  the  self- 
directed  learning  process  (Hannafin  ct  al.,  1994).  The  learner  is  at  the  center  of  the  environment,  both  in  terms  of 
decisions  for  using  it  and  ways  of  making  sense  from  it;  instructor  or  direct  information  is  not  available  to  tell  them 
how  to  interpret  events,  nor  to  monitor  the  strategic  approaches.  Consequently,  learners  invoke  their  own  learning 
strategies  to  organize  and  rearrange  the  learning  process. 

Yet,  implicit  rules  for  using  the  system  also  govern  how  successfully  meanings  will  be  shared.  A traditional 
college  classroom,  for  instance,  has  an  implicit  set  of  rules  defined  for  effective  interaction  --  i.e.,  attend  class,  take 
and  annotate  notes,  use  instructor  office  hours  for  help,  study  for  tests  and  so  on.  Learners  in  such  environments 
have  some  latitude  regarding  how  to  study  or  seek  help  within  the  established  boundaries  (e.g.,  using  study  groups; 
borrowing  notes;  keeping  up  with  assignments);  yet,  choosing  not  to  attend  class  or  study  for  tests,  for  instance, 
represent  approaches  that  would  not  likely  lead  to  success  within  that  given  system.  An  instructor  cannot  help 
students  develop  understanding  if  they  do  not  use  the  opportunities  that  have  been  afforded  them  (e.g.,  attend  office 
hours;  use  class  time  for  clarification).  While  strategics  and  approaches  may  also  depend  upon  the  specific  content 
domain,  the  implicit  rules  of  interaction  must  be  agreed  upon,  shared,  and  understood  by  both  partners  at  the  outset 
(i.e.,  teachers  and  students;  students  and  students;  or  students  and  technology). 

Similarly,  when  learning  with  OLEs,  approaches  must  also  be  consonant  with  the  rules  governing  effective 
use  of  the  environment.  For  many  learners,  these  strategies  and  approaches  may  represent  radical  departures  from 
standard  school-based  learning  activities.  Learners  must  be  able  to  navigate  resources,  recognize  knowledge  gaps 
and  find  relevant  information,  ask  driving  questions,  test  hypotheses,  and  take  action  to  collect  and  analyze 
feedback.  Incongruent  meta-level  approaches  are  significant  in  that  the  system  is  unable  to  discern  whether  suitable 
approaches  are  being  used  — whether  a mouse  click  represents  a random  vs.  reasoned  action  or  whether  tools  are 
being  used  to  test  theories  vs.  see  what  happens. 

In  order  for  meaningful  learning  to  occur,  both  learners  and  the  system  must  be  operating  under  the  same 
set  of  assumptions.  OLEs  require  learners  to  generate  hypotheses  and  to  maneuver  within  the  environment,  even 
when  knowledge  of  its  rules  are  incomplete  (Holland,  et  al.,  1986).  For  instance,  learners  using  a thermodynamics 
microworld  might  manipulate  variables  such  as  surface  area,  surrounding  temperature,  and  insulation  material  of  an 
object.  They  would  use  data  derived  from  taking  action  (e.g.,  testing  out  a hypothesis  that  smaller  objects  wrapped 
in  aluminum  will  retain  heat  better)  as  the  basis  for  addressing  a goal  and  informing  future  action.  Learners  use 
inductive  responses  (Holland  et  al.,  1986),  processing  the  data  that  is  available  to  begin  constructing  and  refining 
models. 

Problems  in  effective  use  of  OLEs  occur  when  learners  use  approaches  that  are  unproductive  within  that 
context  — for  instance,  use  of  traditional  classroom  practices  within  a constructivist  environment.  In  traditional 
environments,  learners  collect  and  learn  information  prior  to  using  it  for  deriving  or  solving  problems,  often  in 
absence  of  a meaningful  or  applied  context.  The  approaches  are  seldom  hypothesis-driven;  rather,  actions  arc 
designed  to  amass  specific  information  until  satisfactory  initial  insights  are  acquired.  Research  on  how  learners  use 
OLEs  indicates  that  they  often  rely  upon  traditional  reasoning  processes  that  arc  unsupported  and  unsuccessful  in  the 
environment  (Land  & Hannafin,  1997;  Wallace  & Kupperman,  1997). 

Wallace  & Kupperman  (1997),  for  instance,  found  that  children  often  applied  a traditional  strategy  of 
“finding  the  answer1’  to  an  inquiry-  and  web-based  context.  That  is,  children  tended  to  view  the  activity  as  finding 
an  answer  to  their  research  question,  and  “thus  reduced  the  task  to  finding  a single  page,  the  perfect  source,  on 
which  the  answer  could  be  found.”  (p.  13).  The  goal  of  these  approaches  was  to  submit  search  after  search  until  the 
smallest  amount  of  hits  were  returned.  The  following  type  of  interaction  reflects  this  traditional  approach: 

“...One  pair  of  students.. .reacted  effusively  to  small  hit  lists,  singing  and  calling  out  ‘yes,  wc  got  it 
now.. .hey  you  guys,  wc  got  it!’  when  they  saw  that  the  number  of  hits  from  a search  was  18,  then  reacting 
with  equivalent  disappointment  when  a cursory  viewing  of  the  hit  list  did  not  reveal  an  obviously 
appropriate  site:  “All  these  things  stink.. .cause  we  put  in  animals  ...  let’s  delete  animals."  Later,  these 
students  produced  a hit  list  with  only  three  pages,  and  ...exclaimed,  “Oh  my  gosh,  wc  got  it!” 


In  this  instance,  learners  seemed  to  rely  on  directed  methods  for  learning  in  contexts  that  did  not  inherently 
support  or  promote  use  of  these  approaches.  When  incongruent  approaches  are  used,  meanings  arc  communicated 
according  to  fundamentally  different  sets  of  assumptions.  Thus,  feedback  is  less  meaningful  as  learners  continue  to 
search  for  specific  answers  or  information  that  will  not  likely  be  rendered  by  the  system.  The  result  is  confusion, 
disorientation,  and  frustration  (Hill  & Hannafin,  1997). 

Conclusions 

This  paper  has  presented  a framework  for  considering  important,  but  often  overlooked,  assumptions 
underlying  use  of  OLEs.  Fundamentally,  we  presume  that  as  learners  use  OLEs,  they  perceive  relevant  information, 
construct  related  interpretations,  and  evaluate  discrepancies  between  their  own  understanding  and  the  meanings 
provided  by  the  system.  The  process  by  which  development  of  understanding  occurs  hinges  upon  a shared, 
reciprocal  and  co-constructive  sense-making  process.  Such  methods,  however,  tacitly  presume  that  learners  possess 
the  self-regulatory  skills  needed  to  make  effective  judgments,  or  can  be  guided  to  make  effective  choices  through 
support  from  the  system.  Yet,  research  has  indicated  that  a convergence  between  initial  learner  models  and  desired 
models  do  not  always  transpire  (Atkins  & Blissett,  1992;  Hill  & Hannafin,  1997;  Land  & Hannafin,  1997).  The 
intention  of  this  paper  was  to  emphasize  the  need  for  more  effective  two-way  communication  of  meanings  between 
the  system  and  learner  and  to  indicate  where  potential  misunderstandings  can  occur. 

During  recent  years,  there  has  been  widespread  interest  in  unleashing  the  capabilities  of  technology  using 
learner-centered  tools  and  environments.  Considerable  disagreement  exists,  however,  related  to  the  usefulness  of 
OLEs  in  supporting  problem-solving  and  learner-centered  inquiry.  The  analysis  presented  in  this  paper,  in  addition 
to  previous  research  on  the  topic  (Hill  & Hannafin,  1997;  Land  & Hannafin,  1997),  have  identified  problems  and 
issues  related  to  design  and  implementation  of  these  systems.  It  is  our  belief  that,  in  order  for  technology-based 
learning  systems  to  evolve,  we  must  begin  to  isolate  what  we  want  to  occur,  how  to  design  systems  to  support  these 
goals,  and  what  appears  instrumental  in  influencing  effective  learning.  Indeed,  many  studies  have  reported  notable 
benefits  of  using  OLEs  to  enhance  student-centered  understanding  (Cognition  and  Technology  Group  at  Vanderbilt, 
1992;  Hard  & Papert,  1991;  Scardamalia  et  al.,  1989).  Yet,  we  also  believe  it  is  equally  important  to  identify  what 
we  don't  want  to  occur,  what  it  looks  like  when  the  process  breaks  down,  and  what  appears  to  be  causing  the 
problems  to  occur.  Thus,  more  complete  understanding  of  the  limitations,  capabilities,  and  assumptions  of  these 
systems  is  needed  in  order  to  capitalize  on  the  unique  potential  of  OLEs. 
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Abstract 

An  abundance  of  information  in  the  classroom  is  conveyed  to  students  through  text-based  resources. 
Written  discourse  is  actually  the  primary  information  source  in  secondary  classrooms.  Research  concerning  text 
processing  and  recall  of  texts  that  are  traditional  in  nature  (e.g..  textbooks,  magazine  articles,  narratives)  has 
highlighted  the  important  influence  of  c*  reader's  prior  knowledge  and  interest  in  a given  domain  on  reading 
performance.  However,  as  the  amount  of  technology  used  for  instructional  purposes  increases,  more  students  will 
increasingly  be  presented  with  computer-based  texts,  such  as  hypertexts . Research  examining  the  roles  of  variables 
like  knowledge  and  interest  in  these  nonlinear  reading  environments  has  been  scarce.  The  present  investigation  was 
undertaken  to  study  how  these  variables  impact  recall  of  information  from  a hypertext  environment.  Results 
indicate  that  domain  knowledge  significantly  predicts  reading  recall.  The  influence  of  topic  knowledge  on  reading 
recall  however  highlighted  differential  prediction  patterns  based  on  the  amount  of  prior  topic  knowledge  the 
individuals  possessed.  Hypertext  navigation,  in  conjunction  with  varying  levels  of  topic  knowledge,  appeared  to 
impact  the  amount  and  type  of  information  recalled. 

An  abundance  of  information  in  the  classroom  is  conveyed  to  students  through  text-based  resources. 
Written  discourse  is  actually  the  primary  information  source  in  secondary  classrooms  (Garner.  1992).  Although 
texts  play  an  important  role  in  the  education  of  students,  not  all  students  reap  the  same  benefits  from  texts.  How 
readers  process  text  may  be,  in  part,  attributable  to  the  amount  the  reader  knows  about  the  topic  or  domain 
represented  in  the  text,  and  how  motivated  or  interested  the  reader  is  to  attend  to  the  information  being  discussed 
(Alexander,  1992;  Alexander,  Kulikowich  & Schulze,  1994). 

Several  different  terms  representing  the  construct  of  knowledge  have  been  presented  in  the  literature  on 
reading  (Alexander,  Schallcrt  & Hare,  1991 ).  Among  these  terms  are  domain  knowledge  and  topic  knowledge.  By 
definition,  domain  knowledge  is  all  known  information  related  to  a field  of  study,  such  as  physics,  mathematics,  or 
psychology  (Alexander,  Pate,  Kulikowich,  Farrell,  & Wright,  1989).  One  consistent  finding  supported  by  the 
literature  is  that  the  more  domain  knowledge  that  one  has,  the  better  one  can  employ  strategies  to  competently 
process  related  text  (Alexander,  1992;  Alexander  & Judy,  1988).  Topic  knowledge , by  contrast,  covers  a much 
smaller  range  than  domain  knowledge.  Topic  knowledge  pertains  to  knowledge  of  specific  concepts  that  are 
encountered  in  text  or  connected  discourse  (Garner  & Gillingham,  1991).  In  general,  what  a reader  knows  about  a 
text  topic  will  determine  what  and  how  much  is  recalled  from  that  text  (Anderson,  1984;  Bransford,  1979). 

Interest  in  a given  domain  has  also  been  found  to  influence  the  amount  of  time  and  attention  a student  will 
devote  to  a particular  reading  exercise  (Garner,  Gillingham  & White,  1989;  Hidi  & Baird,  1986;  Tobias,  1994, 
1995).  Individual  interest  has  been  described  as  interest  in  a domain  or  content  area  (Hidi  & Baird,  1988). 
Alexander  (in  press)  extended  this  definition  to  include  a deep-sealed  investment  in  the  pursuit  of  related 
knowledge.  Research  has  illustrated  that  individual  interest  and  text  recall  have  a linear  relationship  (Schicfclc, 
1991).  That  is,  as  individual  interest  increases,  so  does  recall. 

A second  type  of  interest,  situational  interest , pertains  to  interest  in  specific  features  of  text,  such  as 
pictures  or  illustrations  (Hidi,  1990).  Whereas  individual  interest  is  enduring  in  nature,  situational  interest  is  u 
temporary  arousal,  tied  to  transient  features  of  the  current  situation  represented  in  text  (Anderson,  Shircy,  Wilson  & 
Fielding,  1987;  Kintsch,  1980).  Recent  research  has  shown  that  high  amounts  of  situational ly  induced  interest  can 


he  detrimental  to  reading  recall  (c.g.  inclusion  of  interesting,  but  unimportant  details).  Readers  may  attend  to 
information  that  is  situationally  interesting  but  that  is  of  little  importance  for  understanding  the  content  of  the  text 
(Garner,  1992;  Lawless  & Kulikowich,  1994). 

A large  amount  of  research  has  also  examined  the  interplay  between  the  constructs  of  knowledge  and 
interest  during  the  process  of  reading.  In  a review  of  knowledge  and  interest  literature,  Tobias  (1994)  emphasized 
that, 

it  seems  unrealistic  to  assume  that  there  is,  or  should  be,  little  relationship  between  domain  knowledge  and 

interest.  People  with  high  interest  in  anything  have  probably  acquired  more  knowledge  about  that  subject, 

because  they  are  likely  to  spend  much  more  time  on  activities  related  to  that  field  than  those  less  positively 

disposed  toward  it  (p.  44). 

Statement  of  the  Problem 

The  research  discussed  above  has  only  examined  texts  considered  to  be  traditional  in  nature,  such  as 
magazine  articles,  narratives  or  textbooks.  As  the  amount  of  technology  used  for  instructional  purposes  increases, 
more  students  will  increasingly  be  presented  with  computer-based  texts,  such  as  hypertexts.  Hypertexts  differ 
dramatically  from  traditional  texts  in  that  they  afford  the  reader  the  opportunity  to  interact  with  the  text.  Rather  than 
prescribing  a predetermined  order  in  which  information  is  to  be  acquired  and  comprehended,  as  traditional  texts  do, 
hypertexts  allow  the  reader  to  make  decisions  about  both  what  information  to  access,  and  the  sequencing  of  this 
information  (Landow,  1992).  Essentially,  a reader  must  navigate  the  terrain  of  a hypertext,  thereby  arranging  his  or 
her  own  unique  text. 

Although  the  opportunity  to  select  the  sequencing  and  type  of  information  is  an  advantage  of  hypertexts 
over  traditional  texts,  it  may  also  place  an  added  burden  on  readers.  When  reading  a hypertext,  students  must  be 
able  to  identify  what  information  is  needed  to  enhance  comprehension  and  where  to  find  that  information.  Some 
researchers  have  suggested  that  these  navigational  decisions  may  present  difficulty  to  readers  who  do  not  possess  a 
requisite  amount  of  domain  knowledge  or  topic  knowledge,  or  who  arc  not  interested  in  the  content  area  (i.c.,  lack 
individual  interest)  (Alexander,  Kulikowich  & Jetton,  1994;  Lawless  & Kulikowich,  1994). 

Hypertexts  also  appear  to  have  more  dimensionality  than  do  traditional  texts.  That  is,  hypertexts  include 
many  special  effects  such  as  digitized  movies,  sound  and  visual  effects,  and  graphics.  These  special  features  appear 
to  enhance  hypertext,  making  it  richer  and  more  complex  (Carver,  Lehrer,  Connell,  & Erickson,  1992),  and  may  also 
increase  situational  interest.  However,  this  heightened  slate  of  situational  arousal  may  cause  students  to  seek  out 
these  special  features,  perhaps  at  the  expense  of  the  instructional  content  (Lawless  & Kulikowich,  1994).  As  such, 
it  appears  that  the  amount  of  situational  interest  an  individual  has  in  the  computerized  environment  and  its  special 
features  may  also  affect  navigation  and  recall. 

The  purpose  of  this  investigation  was  to  extend  the  findings  of  prior  research  on  knowledge  and  interest  in 
traditional  text  processing  to  more  deeply  examine  the  role  of  these  variables  in  hypertext  processing.  Specifically, 
this  study  examined  the  relationships  among  domain  knowledge,  topic  knowledge,  individual  interest  and  situational 
interest  within  the  context  of  a hypertext  environment.  Additionally,  the  relationship  among  these  variables  and  text 
recall  was  examined. 

Methodology  and  Procedures 

Participants  in  this  study  included  107  undergraduate  and  graduate  students  enrolled  in  the  School  of 
Education  at  a large  Northeastern  university.  This  pool  of  participants  was  chosen  because  it  was  deemed  that  these 
participants  would  have  varying  knowledge  levels  of  the  basic  psychology  topics  that  represent  the  information  in 
the  materials  used  in  this  study.  Further,  this  sample  of  individuals  comprised  a group  of  students  who  were 
considered  competent  readers  (i.c.,  as  determined  by  academic  performance  and  university  enrollment 
qualifications).  Additionally,  not  one  of  the  participants  indicated  a reading  disability  that  might  interfere  with 
participation  in  this  study. 

Materials  for  this  investigation  included  the  following:  I)  nine-item  open  ended  assessments  of  domain 
knowledge;  2)  nine-item  assessment  of  individual  interest;  3)  nine-item  assessment  of  situational  interest;  4)  a 150 
card  hypertext  document;  5)  a structured  recall  assessment;  and,  6)  an  unstructured  recall  assessment.  After 
adjusting  for  attenuation,  reliabilities  for  knowledge  and  interest  measures  all  exceeded  .75.  Interratcr  agreement 
concerning  the  scoring  of  these  assessments  were  better  than  85%  in  all  cases. 

Students  were  first  administered,  in  counterbalanced  order,  the  knowledge  and  interest  assessments.  After 
completing  these  measures,  students  were  then  given  a brief  overview  and  tutorial  about  the  hypertext  environment 


252  253 


including  information  on  how  they  could  access  subfields  of  the  text  and  revisit  previous  sections  of  the  text  as  they 
saw  necessary.  Students  were  instructed  to  study  the  text  and  self-selected  subfields  as  carefully  as  they  could  for 
they  would  be  asked  a series  of  questions  at  the  end  of  their  session.  These  questions  were  designed  to  direct 
students  to  recall  as  much  of  the  information  from  the  text  as  they  could. 

Results 

A series  of  standard  multiple  regression  analyses  was  performed  to  analyze  the  relationships  among 
knowledge,  interest  and  recall  constructs.  The  use  of  multiple  regression  was  warranted  in  this  investigation 
because  the  examination  of  the  partial  correlation  coefficients  allows  for  the  interpretation  of  the  relationships 
between  independent  variables  and  the  corresponding  dependent  variable  while  holding  all  other  variables  in  the 
analysis  constant  (Tabachnick  & Fidel  1,  1989).  As  such,  the  partial  correlation  coefficients  reveal  the  unique 
relationships  between  constructs  by  parsing  out  the  variance  shared  by  other  variables  in  the  analysis. 

Results  indicated  a significant  parital  correlation  between  domain  knowledge  and  individual  interest  (r=.28, 
pc. 05),  after  adjusting  for  attenuation.  However,  no  significant  relationship  was  identified  between  domain 
knowledge  and  situational  interest.  When  recall  measures  were  used  as  the  dependent  variables,  domain  knowledge 
was  found  to  be  significant  predictor  of  both  structured  and  uncfructured  recall  * 'rtial  r=.285,  pc.05,  partial  r=.414, 
pc.Ol,  respectively).  However,  when  topic  knowledge  was  used  as  the  predictor  of  recall  performance,  the  topic 
area  were  participants  had  more  prior  knowledge  was  a significant  predictor  of  structured  recall  (t96=2.54,  pc.Ol), 
whereas  the  lower  knowledge  topic  area  predicted  unstructured  recall  (t96=2.36,  pc.05).  Neither  interest  measure 
significantly  predicted  reading  recall  from  the  hypertext  document. 

These  results  concur,  in  part,  with  prior  literature  on  traditional  reading  environments.  That  is,  domain 
knowledge  has  a powerful  influence  on  the  amount  of  information  recalled  regardless  of  the  media  used  to  present 
the  text.  However,  the  lack  of  ability  to  identify  a significant  relationship  between  interest  and  reading  recall,  in 
conjunction  with  the  differential  pattern  of  recall  associated  with  topic  knowledge  areas  calls  into  question  whether 
students  use  the  same  strategies  when  acquiring  information  from  hypertexts  as  they  use  with  traditional  texts. 
Review  of  the  navigational  data  of  participants  through  information  space  afforded  by  the  hypertext  highlights  that 
the  novel  relationships  among  knowledge,  interest  and  recall  constructs  in  this  study  might  be  attributable  the 
various  methods  of  navigation  selected  by  participants.  It  appears  that  higher  knowledge  learners  selected  and 
sequenced  information  more  efficiently,  thereby  facilitating  text  recall.  However,  low  knowledge  readers  seemed  to 
have  difficulty  making  associations  between  informational  units  from  the  hypertext  environment.  Additionally, 
these  readers  allocated  more  time  to  the  special  features  of  the  text  (c.g.,  movies,  sound  effects...)  -often  times  at  the 
expense  of  more  important  informational  content. 

Educational  and  Scientific  Implications 

Studying  hypertext  processing  is  not  a simple  undertaking.  There  arc  many  variables  to  consider  and  many 
types  of  technological  features  that  can  be  studied.  However,  as  the  information  age  continues  to  present  learners 
with  environments  that  house  an  abundance  of  materials  and  resources,  it  would  seem  most  important  to  explore  the 
relationships  among  variables  that  will  facilitate  human  processing.  Navigational  strategics  appear  to  be  critical  to 
the  effective  processing  of  computer-based  texts  such  as  hypertexts.  As  such,  researchers  need  to  continue  to 
address  the  role  of  navigational  strategies  and  the  interaction  of  these  strategics  with  cognitive  and  affective 
variables. 
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Abstract 

The  purpose  of  this  study  was  to  investigate  the  effectiveness  of  narrative  versus  step-by-step  instructions  for  a 
computer  task  Naive  participants  were  observed  as  they  completed  the  steps  for  retrieving . replying,  spell  checking,  and 
sending  email  using  software  on  a mainframe  computer.  There  was  no  performance  time  difference  between  the  two 
groups,  but  the  step-by-step  treatment  made  fewer  errors  during  (he  more  complex  tasks. 

One  problem  often  slighted  in  the  software  development  process  is  the  development  of  appropriate 
documentation  fnr  the  user.  This  observation  is  supported  by  the  growing  number  of  after-market  software  books  and  the 
growing  need  for  software  technical  support.  Although  there  are  many  examples  of  excellent  software  manuals,  there  are 
also  numerous  examples  of  manuals  that  offer  little  more  than  a programmer’s  description  of  how  the  software  works. 
An  analysis  hy  Kaplan  (1989)  revealed  five  types  of  software  documentation:  reference  manual,  tutorial  manual,  user  (or 
combination)  manual,  quick  reference  card,  and  on-line  help.  She  believed  that  for  novices,  . . the  tutorial  is  the  most 
essential  piece  of  documentation”  (p.32).  One  problem  with  tutorials  is  the  writer’s  potential  for  underestimating  the 
intelligence  of  the  user  and  sinking  to  the  “Dick  and  Jane”  level  (Deterline,  1988). 

McGehee  (1984)  defined  tutorials  as  “a  cookbook,  hand-holding,  step-by-step  method  of  teaching  the  concepts 
and  techniques  of  using  a program”  (p.57).  It  is  interesting  to  note  that  McGehee’s  definition  uses  the  term  step-by-step, 
which  suggests  a non-narrative  approach.  Tutorials  are  recommended  by  McGehee  if  the  manual  is  the  user's  first 
introduction  to  a program,  if  the  program  is  complex,  and  if  the  user  is  a computer  novice.  The  major  problem  identified 
with  writing  tutorials  is  trying  to  match  the  user's  knowledge,  a problem  that  depends  on  the  particular  software  and  the 
intended  audience.  Deterline  (1988)  issued  a plea  for  inclusion  of  user  needs  when  documei nation  is  written,  because 
much  so ftware  documentation  is  written  by  subject-matter  experts  (SME),  often  the  computer  programmers  and  it  is  very 
difficult  for  non-experts  to  comprehend. 

If  the  documentation  manuals  are  the  user's  first  contaci  with  the  software,  how  should  the  information  be 
presented?  Brockmann  (1990)  discussed  the  limitations  of  tutorials  in  general,  which  he  believes  may  constrain  users  to 
a set  of  minimum  expectations.  Tutorial  material  is  “usually  organised  around  user  tasks  or  around  a hierarchy  of  user 
needs”  (p.  74)  In  contrast,  Thurston  (1986)  slates  that  general  purpose  computer  programs  (c.g.  programming 
languages)  are  better  served  with  narrative  form  tutorials,  while  narrowly  defined  programs  (c.g.  accounting  packages) 
are  best  dealt  with  using  step-by-step  tutorials.  Webb  (1989)  also  recommended  step-by-step  tutorials  for  procedures  and 
narrative  form  for  overviews  or  introductions.  “Cookbook”  tutorials  which  consist  of  simple  action  sentences  (c.g.  Turn 
on  computer.  Click  on  icon  for  program.  Select  word  processing)  differ  in  form  from  numbered  steps  which  move  users 
sequentially  through  a procedure  tutorial  (e.g.,  1 . Turn  on  computer.  2.  Locate  the  program’s  icon.  3.  Double  click  on  the 
icon.  4.  When  the  program  starts,  find  the  box  for  word  processing  and  click  on  it).  Tutorials,  regardless  of  form  should 
begin  with  a general  overview  and  include  as  many  illustrations  as  possible.  However,  Kaplan  (1989)  suggests  that 
narrative  style  instruction  often  hides  important  messages  in  a mass  of  text.  Although  the  use  of  numbered  steps  makes 
logical  sense  for  complex  tasks,  a review  of  the  instructional  design  literature  failed  to  identify  any  research  comparing 
narrative  versus  step-by-step  approaches  to  leaching  procedures. 

The  purpose  of  this  study  was  to  investigate  the  effectiveness  a narrative  versus  step-by-step  presentation  of  a 
procedure  for  naive  computer  users.  It  was  predicted  that  the  step-by-step  approach  would  be  more  time  efficient,  since  a 
narrowly  focused  application  (email)  was  used  in  the  study.  It  was  also  predicted  that  participants  receiving  the  step 
treatment  would  make  fewer  overall  errors  and  fewer  errors  of  each  type.  Wc  were  also  interested  in  knowing  how  the 
two  treatments  influenced  participants’  attitudes  toward  the  instructional  materials. 
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Method 

Participants 

The  participants  in  this  study  were  31  undergraduate  education  students  enrolled  in  a computer  literacy  course 
at  the  University  of  Memphis  during  the  Summer,  1996  semester.  The  participants  generally  had  little  computer 
experience  and  many  suffered  from  “computerphobia.’'  None  of  the  participants  had  any  prior  knowledge  of  email. 
Participants  were  randomly  assigned  to  one  of  the  two  treatment  groups. 

Materials 

The  treatment  material  consisted  of  a three  part  instructional  unit  providing  information  on  how  to  log  onto  the 
University  EMAIL  system,  read  and  compose  email,  spell  check,  reply  to,  and  send  messages.  The  instructions 
encouraged  students  to  proceed  at  their  own  pace. 

Lesson  Description 

A unit  of  instruction  was  on  how  to  use  the  University’s  VAX  EMAIL  server,  compose,  read,  spell  check,  reply 
to,  and  send  EMAIL  messages  using  the  VAX  software.  The  first  treatment  group  was  the  narrative  treatment  (n=l5). 
TIi is  treatment  group  read  text  presented  in  a traditional  narrative  format  with  each  paragraph  including  from  three  to 
seven  steps.  The  following  is  a sample  of  the  narrative  treatment. 

Turn  on  your  computer.  After  the  computer  has  finished  its  start-up  process,  you  will  see  the  “Finder”  screen. 
Use  the  mouse  to  move  the  arrow  to  the  Apple  icon  (picture)  in  the  top  left  corner  of  the  screen.  With  the  arrow 
pointing  at  the  Apple  icon,  press  the  mouse  button  and  hold  it  down. 

The  second  treatment  group  (n=16),  step-by-step,  read  instructions  that  were  clearly  enumerated  and  separated 
from  each  step.  The  following  is  a sample  of  the  step-by-step  treatment. 

1 . Turn  on  your  computer. 

2.  After  the  computer  has  finished  its  start-up  process,  you  will  see  the  “Finder”  screen. 

3.  Use  the  mouse  to  move  the  arrow  to  the  Apple  icon  (picture)  in  the  top  left  corner  of  the  screen. 

4.  With  the  arrow  pointing  at  the  Apple  icon,  press  the  mouse  button  and  hold  it  down. 

The  two  treatments  contained  the  same  information  but  varied  in  the  way  it  was  displayed  on  the  page.  The  only 
difference  was  the  numbering  of  the  steps  in  the  second  treatment. 

Instruments 

After  completion  of  their  respective  treatments,  participants  were  given  a survey  designed  to  assess  their 
attitudes  toward  the  instruction.  The  attitude  survey  consisted  of  six  core  items  that  asked  participants  to  rate  the  lesson, 
computer  learning,  the  VAX  system,  the  usefulness  of  email,  this  lesson,  the  sufficiency  of  items,  and  future  email  use 
using  a five  point  Likert  scale  (l=strongly  disagree,  5=strongly  agree).  The  second  instrument  was  an  observation  form 
used  by  the  experimenter  during  the  treatment.  The  form  listed  each  step  of  the  process  with  places  to  indicate  if  the  step 
was  performed  correctly  and  in  sequence,  performed  incorrectly,  performed  correctly  out  of  sequence  or  skipped.  There 
was  also  a place  for  noting  the  starting  and  ending  times  and  a section  for  noting  participant  behaviors  (see  Figure  1 ). 
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Procedure 

Participants  were  scheduled  to  complete  the  instruction  in  a onc-on-one  setting  with  the  experimenter.  Each 
was  seated  at  a color  Macintosh  computer  (LCII  or  LCIII)  and  given  the  printed  instructions.  The  starting  and  ending 
limes  were  noted  for  each  participant.  The  experimenter  observed  each  subject  complete  die  tasks.  The  sequence  of  steps 
completed  and  the  type  of  errors  made  were  recorded  by  the  experimenter  using  the  following  instrument.  Each  time  a 
participant  attempted  a step,  the  participant’s  action  was  evaluated  and  recorded  on  the  checklist. 

Results  and  Discussion 

The  first  analysis  was  a comparison  of  completion  limes  and  overall  errors  for  each  of  the  treatments  with  T- 
tesis  and  using  SPSS  statistical  software.  No  significant  differences  were  found  for  time  on  task  (times  ranged  from  19 
to  23  minutes),  or  overall  errors  by  treatment  (overall  errors  ranged  from  0 to  4). 

The  errors  in  the  13  steps  in  the  procedure  were  summed  into  four  g'  ;s:  Steps  1-4  (initial  logon);  Steps  5-7 
(composing  and  sending  message);  Steps  8-10  (logon  and  reading  message);  uud  Steps  11-13  (replying,  spell  checking 
and  sending  message).  C ily  the  final  group  of  steps  varied  significantly  by  treatment  when  compared  using  MANOVA, 
with  the  step  procedure  i oup  making  fewer  errors  (F=5.0O24). 

Table  I.  Errors  on  Task 


Variable 

Treatment  #1  (Narrative) 

Treatment  #2  (Step) 

F.  Prob. 

A lean 

5.  D . 

Mean 

S.D. 

p<.05 

Error  Steps  1-4 

.200 

.414 

.563 

.814 

.1329 

Error  Steps  5-7 

.600 

.632 

.313 

.479 

.1626 

Error  Steps  8-10 

.067 

.258 

.000 

.000 

.3097 

Error  Steps  11-13 

.533 

.640 

.125 

.342 

.0332 

When  type  of  errors  by  treatment  were  compared  with  MANOVA,  only  the  Wrong  Command  Key  errors 
differed,  with  the  step  procedure  treatment  group  making  significantly  fewer  errors  (F=7.1 382). 
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Table  2.  Errors  by  Treatment 


Variable 

Treatment  #1  (Narrative) 

Treatment  #2  (Step) 

F.  Prob. 

Mean 

S.D . 

Mean 

S.D. 

p<.05 

Wrong  Command  Key 

.733 

.704 

.188 

.403 

.0122 

Wrong  Entry  Typed 

.400 

.632 

.750 

1.065 

.2790 

Wrong  Sequence 
Worked 

but  .133 

.352 

.063 

.250 

.5212 

Wrong  Sequence  Error 

.200 

.414 

.000 

.000 

.0629 

MANOVA  comparisons  found  no  other  significant  differences  between  groups  for  Looks  at  Instructions,  Asks 
for  Help,  or  Shows  Frustration.  The  attitude  survey  questions  also  revealed  no  significant  differences  between 
treatments,  although  both  treatment  groups  recorded  positive  overall  attitudes  toward  the  instruction. 

The  lack  of  significant  differences  in  times  for  the  treatment  groups  may  have  resulted  from  a motivated 
participant  group.  All  participants  were  required  to  learn  to  use  email  in  their  ongoing  literacy  course  which  may  have 
resulted  in  a greater  degree  of  learner  engagement  than  might  have  otherwise  occurred.  Thus  there  was  no  support  for  the 
first  hypothesis  (the  step  approach  will  be  more  time  efficient).  This  finding  is  in  agreement  with  a study  by  Morrison, 
Ross,  Case  and  Gopalakrishnan  (1995)  which  found  that  purposeful  engagement  raised  scores. 

Similarly,  there  were  no  significant  differences  in  overall  errors  by  treatment  in  this  study.  This  lack  of 
difference  may  be  due  to  the  relative  simplicity  of  the  procedure,  with  few  opportunities  for  errors.  However,  there  were 
significant  differences  in  performance  on  the  most  difficult  part  of  the  procedure  (replying  to  a message,  spell  checking 
and  sending  the  reply).  The  step-by-step  group  made  fewer  keystroke  errors.  This  finding  suggests  that  step-by-step 
procedures  may  produce  fewer  errors  when  the  procedure  is  more  complex.  Participants  using  the  narrative  approach 
found  this  group  of  steps  very  difficult  and  often  showed  frustration  while  attempting  to  complete  the  tasks.  The  step-by- 
step  group  appeared  to  have  gained  enough  confidence  in  their  ability  that  they  were  not  as  intimidated  by  this  portion  of 
the  lesson. 

Significant  differences  were  also  found  for  Wrong  Command  Sequence  errors.  A Wrong  Command  Sequence 
error  occurs  when  a participant  presses  the  wrong  key  and  is  not  successful  at  completion  of  the  step.  This  result 
appeared  to  be  due  to  a loss  of  concentration  by  participants  receiving  the  narrative  treatment.  These  participants  often 
lost  their  place  in  the  procedure  because  they  lacked  the  number  cues  provided  to  the  step-by-step  participants.  This 
finding  is  consistent  with  Thurston’s  (1986)  recommendation  that  step-by-step  procedures  are  more  appropriate  for 
narrowly  defined  applications  such  as  email. 

The  lack  of  differences  in  attitudes  by  treatment  may  be  explained  by  the  participants’  need  to  acquire  the  skill 
taught  by  the  lesson.  Some  negative  behaviors  were  observed  during  the  treatments  (mostly  impatience  over  having  to 
read  the  instructions),  but  the  majority  of  the  participants  were  positive  (and  often  enthusiastic)  about  their  new  skill. 
Another  factor  that  may  have  influenced  performance  and  attitudes  was  the  individualized  testing  situation.  This 
approach  was  in  direct  contrast  to  the  20:1  student-instructor  ratio  in  the  computer  literacy  class.  This  onc-on-onc 
environment  may  have  increased  their  confidence.  Also,  the  treatment  instructions  were  considerably  more  detailed  than 
those  provided  in  the  textbook. 

Most  of  the  students  (n=28)  were  forced  to  seek  help  from  the  experimenter  at  least  once  because  of  confusion 
over  the  procedure,  especially  when  asked  to  choose  a new  password  (problems  included  new  password  already  in  use 
and  illegal  password  selected).  Almost  half  (n=13)  showed  signs  of  frustration,  especially  when  trying  to  press  the 
CONTROL-Z  key  combination,  which  is  a unique  keystroke  on  a Macintosh.  No  participants  attempted  to  explore  the 
menus  or  other  options  and  only  one  participant  tried  to  send  any  additional  messages. 

It  appeared  from  the  observations  that  the  major  determining  factor  of  success  was  the  participant’s  motivation, 
not  the  format  of  the  written  procedure.  Participants  who  experienced  difficulty  appeared  to  skip  portions  of  the  printed 
lesson  and  skimmed  longer  explanations  which  resulted  in  less  understanding  and  more  errors.  Participants  with  higher 
motivation  completed  the  tasks  with  few  mistakes  and  displayed  interest  in  the  underlying  structure  of  email.  Those  with 
lower  motivation  often  ignored  instructions  (especially  concerning  spell  checking)  and  showed  little  overall  interest. 

The  differences  in  Control  Key  sequence  errors  indicated  that  the  step  procedure  produced  fewer  errors.  One 
possible  explanation  is  that  the  more  complex  the  task  and  the  more  critical  the  order  of  the  steps,  the  more  advantageous 
are  the  step  procedures. 
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Overall,  the  results  were  less  conclusive  than  predicted,  but  significant  differences  in  two  of  the  variables 
examined  (Steps  11-13  and  Wrong  Command  Sequence)  indicates  that  there  are  measurable  differences  between  the  two 
methods  of  instniction. 

Recommendations 

The  study  indicated  that  differences  in  effectiveness  of  step-by-step  procedures  may  be  less  than  generally 
believed.  The  results  of  the  study  suggests  additional  research  with  complex  procedures  should  be  used  to  determine  if 
complexity  of  task  plays  a part  in  differentiating  methods  of  instruction.  Future  studies  should  include  a variety  of  tasks 
that  are  simple  and  complex,  short  and  long,  and  familiar  and  unfamiliar  to  the  participant. 
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Abstract 

Educators  have  not  yet  clearly  described  effective  methods  for  implementing  and  managing  performance- 
based  learning  experiences  that  incorporate  electronic  technology.  While  we  have  recognized  the  benefits  of 
constructivist  approaches  to  learning , the  literature  says  little  about  the  role  of  the  teacher  within  computerized 
learning  contexts.  This  paper  reports  on  two  research  studies.  The  first  research  study  investigates  teachers 
monitoring  strategies  during  computer-assisted  composition  instruction.  The  findings  reveal  four  principles 
underlying  a strategic,  proximal  instructional  process.  The  principles  are  collaborative  assessment,  guided 
practice,  instructional  branching,  and  learner  self-monitoring  skills’  development.  The  computer  is  described  as  a 
cognitive  tool  supporting  and  facilitating  teachers’  active  involvement  in  students’  actual  writing  process  which 
represents  a change  in  process  writing  pedagogy  found  in  non-compute rized  settings  focused  on  analysis  of 
students’  writing  products,  albeit  drafts.  The  second  study  reports  on  a related  issue  concerning  the  design  of 
electronic  performance-based  assessment  systems.  The  findings  are  a set  of  guidelines  instructional  designers  and 
educators  can  use  in  planning  for  the  use  of  electronic  assessment  systems  for  educational  purposes.  Both  studies 
point  to  the  need  for  technological  solutions  to  provide  efficient,  valid , and  reliable  information  available  to 
teachers  and  learners  to  enhance  the  types  of  instructional  processes  that  accompany  active , engaged  learning 
experiences . 

Background  and  Theoretical  Perspective 

Educational  reform  measures  call  for  authentic  learning  contexts.  Technology  lias  the  potential  to  facilitate 
and  support  authentic,  performance-based  learning  experiences  through  its  capacity  to  interactively  engage  learners. 
Educators,  however,  have  not  vet  clearly  described  effective  instructional  methods  for  implementing  and  managing 
performance-based  learning  experiences  that  incorporate  electronic  technology.  While  wc  have  recognized  the 
benefits  of  constructivist  approaches  to  learning,  the  literature  says  little  about  role  of  the  teacher  within 
computerized  learning  contexts.  In  a seminal  work,  Collis  (ct.al.,  1996)  reported  findings  from  three  international 
research  studies  focused  on  computers  and  learning  across  multiple  contexts.  In  the  analysis  of  the  data,  the  teacher 
was  identified  as  the  critical  influence  in  enhancing  students’  learning  regardless  of  many  variables  at  the  classroom, 
hardware,  and  software  levels. 

This  paper  reports  on  two  studies  addressing  issues  related  to  performance-based  learning  and  uses  of 
technology  to  support  it.  The  first  study  describes  findings  from  a grounded-theory  research  study  focused  on 
writing  teachers’  monitoring  activities  in  a computerized  learning  environment.  The  findings  describe  a set  of 
instructional  strategies  that  facilitate  proximal  instruction  and  its  implications  for  authentic,  performance-based 
learning  experiences.  The  second  study  describes  the  development  of  guidelines  for  using  technology  to  help 
manage  the  types  of  data  generated  from  performance-based  learning  experiences.  Standardized  assessment  methods 
aic  inadequate  for  tracking  and  reporting  educational  progress  found  in  performance-based  approached  to  education. 
That  is  not  to  say  all  standardized  testing  is  out,  and  alternative  assessment  is  in.  Rather,  constructivist  approaches 
to  learning  call  for  the  use  of  alternative  assessment  measures  1)  to  develop  the  learner's  cognitive  strategies  for  self- 
monitoring  of  progress  2)  to  foster  the  learner's  ability  for  higher-order  thinking  skills,  3)  to  measure  progress 
against  the  learner’s  own  development,  not  the  norm,  and  4)  to  provide  more  accurate  evidence  of  a learner's  abilities 
than  traditional  tests  (Boyer,  1995;  Cole,  Struyk,  Kinder,  Sheehan,  and  Kish,  1997;  Wiggins,  1993). 

The  educational  reform  movement  has  been  strongly  fell  across  educational  contexts.  The  general  public 
wants  students  to  be  prepared  to  meet  the  challenges  inherent  in  our  complex  and  fast-paced  culture,  challenges  that 
arc  often  driven  by  technology.  Learning  content  and  context  is  coming  under  careful  examination  to  evaluate  the 


261  2C1 


relevancy  and  applicability  for  preparing  children  to  live  in  a high-tech  society.  Our  technological  society  requires 
higher-order  thinking  skills  such  as  interpreting,  evaluating,  synthesizing,  and  communicating  new  knowledge; 
problem  solving;  collaborative  teamwork;  goal  setting;  and  decision  making.  Students  need  to  learn  how  to  monitor 
their  own  thinking  processes  and  how  to  use  technological  systems  to  manage  and  convey  information.  The  research 
discussed  in  this  paper  provides  a rationale  for  using  proximal  instruction  strategies  and  technological  management 
tools  to  support  learner-centered  principles  in  education.  Learner-centered  principles,  which  underlie  constructivist 
approaches  to  education  (McCombs,  ct  al,  1993),  claim  that  learning  which  is  closest  to  the  self  is  the  most 
powerful.  This  paper  provides  insight  into  monitoring  strategies  and  technology  guidelines  that  can  assist  teachers 
and  student  with  individualizing  instruction  and  assessment  as  well  as  fostering  student  accountability  for  learning. 

Investigating  Teachers*  Monitoring  Strategies 

The  literature  reflects  a historical  problem  with  technology  implementation  efforts.  The  problem  is  that 
curriculum-making  power,  historically  vested  in  the  teacher,  becomes  vested  in  the  technology  system  and  overrides 
the  teacher’s  decision-making  abilities,  which  represents  a deskilling  effect  (Streibel,  1993).  Yet  the  teacher  is  still 
responsible  for  implementation  choices  al  this  point  in  lime.  Knupfer  (1993)  argues  that  "the  success  of  any  such 
implementation  effort  will  depend  largely  on  the  teachers  who  determine  the  daily  school  activities"  (p.  164). 
Knupfer  criticizes  research  agendas  that  focus  exclusively  on  the  dynamics  between  the  learner  and  technology.  She 
also  points  out  that  research  about  the  teacher's  role  in  implementation  has  received  little  attention,  despite  "an 
overwhelming  body  of  evidence  [that]  points  to  the  centrality  of  teachers  in  the  process  of  educational  innovations" 
(p.  165). 

In  regard  to  the  implementation  of  computer-assisted  composition  instruction,  Selfe  (1990)  writes  that  the 
computer-assisted  composition  instruction  movement  has  been  accompanied  by  a pioneering  spirit  that  "remains 
atheoretical,  untested,  unexamined,  and  less  than  systematic"  (p.  191).  Likewise  Reinking  and  Bridwcll-Bowics 
(1991)  claim  that  "no  comprehensive  computer-hased  writing  curricula  have  been  developed"  (p.  314).  The  research 
conducted  in  the  field  of  computer-assisted  college  composition  instruction  reflects  the  broader  approach  of 
technology  implementation  efforts-a  focus  on  learner-technology  dynamics  at  the  exclusion  of  learner-teacher 
dynamics.  Curtis  and  Klem  (1992)  echo  the  concern  for  a lack  of  research  about  teachers  in  computerized  teaching 
environments: 

Our  own  readings  have  found  a corresponding  absence  of  teachers  and  actual  teaching  from  the  bulk  of 
computer  research,  even  from  examination  of  the  computer's  relationship  to  teaching  methodology;  the 
focus,  rather,  seems  inevitably  to  shift  to  the  presence  of  the  machines.  Wc  fear  that  this  shift  denotes  a 
somewhat  odd  substitution  of  computer  for  teacher  as  locus  of  influence,  instruction  and  motivation  among 
student  writers  (p.  1 57). 

The  problem  with  the  shift  is  that  computer  systems  and  programs  arc  not  a substitute  for  social  interaction 
processes  relevant  to  the  teaching  of  writing.  During  the  writing  process  the  writer  acts  “as  a creator  of  meaning” 
(Berlin,  1994,  p.  19).  In  addition,  the  substance  of  the  writing  embodies  differences  in  human  experiences,  values, 
purposes  and  goals.  Berlin  points  out  the  pedagogical  value  put  on  fostering  writers’  individuality. 

This  section  of  this  paper  reports  on  a research  study  conducted  by  McNabb  (1996)  that  had  two  primary 
purposes.  First,  its  purpose  was  to  describe  the  monitoring  activities  engaged  in  by  composition  teachers  who  are 
involved  in  implementing  computer-assisted  composition  instruction.  The  second  purpose  was  to  articulate  the 
pedagogical  principles  underlying  the  monitoring  activities  of  teachers  in  the  context  of  computer-assisted 
composition  instruction.  The  researched  focused  on  the  monitoring  activities  of  teachers  during  computer-assisted 
composition  instruction  with  the  assumption  that  teachers’  monitoring  strategics  were  key  indicators  of  what 
facilitates  writing  skill  development  in  student  writers  during  computer-assisted  writing  activities.  This  premise 
arose  from  a body  of  literature  about  the  cognitive  writing  process  grounded  in  the  seminal  research  conducted  by 
Hayes  and  Flower  (19X0).  Hayes  and  Flower  identified  a “moniior”  as  a component  of  the  cognitive  writing 
process;  the  monitor  was  believed  to  facilitate  recursion  between  the  planning,  production,  ?nu  revision  stages  of  the 
process  writing  model.  While  those  in  the  field  of  English  composition  have  conducted  a vast  amount  of  research 
regarding  other  subcomponents  of  the  cognitive  writing  process  model  developed  by  Hayes  and  Flower,  the 
monitoring  component  of  their  model  has  gone  largely  unresearched.  Nonetheless,  others  have  categorized 
monitoring  as  mctacognilion  inherent  in  the  writing  process  (Flavcll,  1987;  Perkins,  Simmons,  & Tishman,  1990; 
Scardamalia,  Bcrcitcr,  & Steinbacl,  1984).  Thus,  the  research  represents  an  original  approach  to  studying  the 


cognitive  writing  process  by  focusing  on  the  monitoring  activities  that  teachers  engage  in  with  student  writers  during 
computer-assisted  instruction. 

Two  central  questions  guided  the  investigation:  1)  What  are  the  characteristics  of  composition  teachers' 
monitoring  activities  in  a computer-networked  learning  environment?,  and  2)  What  is  the  nature  of  the  association 
between  the  teachers’  monitoring  activities  and  students'  computer-assisted  writing  activities? 

Research  Method 

The  researcher  employed  grounded-theory  methodok  ;y  to  gather  data  about  the  monitoring  activities  of 
teachers  during  computer-assisted  composition  instruction.  t . grounded-theory  approach  was  used  to  inductively 
construct  a theoretical  framework  for  interpreting  the  social  phenomena  understudy  (Corbin  & Strauss,  1990; 
Glaser  1978,  1992;  Glaser  & Strauss,  1967;  Strauss,  1987;  Strauss  & Corbin,  1990,  1994).  Research  derived  from 
grounded-theory  methodology  typically  provides  insight  into  “patterns  of  action  and  interactions  between  and 
among  various  types  of  social  units  (i.e.,  actors)’’  (Strauss  & Corbin,  1994,  pg.  278).  The  patterns  of  action  and 
interaction  between  teacher  and  student  and  the  underlying  conceptual  processes  were  the  focal  point  of  this 
research  project;  hence,  grounded-theory  methodology  suited  the  research  project  well.  The  primary  purpose  of  the 
study  was  to  describe  and  explain  teachers’  monitoring  practices  during  computer-assisted  composition  instruction. 
The  initial  working  definition  of  monitoring  was  any  interaction  between  teacher  and  student(s)  during  the 
computer-assisted  instructional  sessions. 

The  research  was  conducted  through  the  English  Department  at  a mid-sized  public  university  representative 
of  mainstream  American  institutions  of  higher  education.  The  English  Department  advocated  process-writing 
pedagogy  in  teaching  the  traditional  freshman  composition  sequence.  The  English  Department  also  had  a 10-ycar 
history  of  implementing  computer-assisted  instruction  (CAT)  in  the  composition  course.  The  year  this  study  was 
conducted,  the  university  had  implemented  CAX  into  100%  of  the  freshman  two-semester  composition  sequence. 
The  study  occurred  in  the  last  six  weeks  of  the  second  semester  of  the  two-course  sequence.  Nine  teachers  and  their 
180  students  were  selected  for  the  study.  The  teachers  played  three  roles  in  the  study.  First,  the  teachers  were 
subjects  of  the  researcher’s  observation  and  inquiry.  Second,  they  took  on  the  role  of  participant  observer.  Third, 
they  served  as  key  informants  for  collecting,  analyzing,  and  reporting  data.  In  order  to  answer  the  research 
questions,  the  researcher  designed  and  used  a screening  questionnaire  to  select  the  participant  teachers  and  designed 
the  initial  procedures  and  instruments  for  data  collection  based  on  perspectives  from  the  technical  literature  in 
instructional  technology,  educational  psychology,  and  composition  pedagogy.  Teachers  selected  for  participation  in 
the  study  held  important  similarities  in  their  teaching  practices  and  heliefs  about  the  writing  process  and 
instructional  uses  of  the  computer,  according  to  the  screening  questionnaire.  Thus,  the  data  represents  the 
monitoring  practices  of  a relatively  homogeneous  group,  which  allowed  for  the  rich  variation  in  monitoring 
strategics  used  in  computer-assisted  process  writing  to  emerge.  After  selecting  the  participant  teachers,  one  of  each 
teacher’s  two  assigned  course  sections  was  randomly  selected  for  the  study.  The  level  of  writing  skill  among  the 
student  groups  was  assumed  to  be  similar,  since  students  were  pre-tested  by  the  university  and  placed  into  writing 
classes  according  to  similar  writing  skills. 

In  order  to  answer  the  stated  research  questions  and  to  add  strength  to  the  design  of  the  study,  triangulation 
of  data  collection  occurred  in  a naturalistic  setting.  Data  sources  included  observation  of  45  CAI  sessions  and  in- 
depth  beginning  and  exit  interviews  with  participant  teachers.  The  teachers  tape-recorded  their  naturally  occurring 
df.logue  with  students  during  the  CAI  sessions.  They  captured  students’  beginning  and  ending  electronic  files  on 
the  computer  network  and  transferred  the  files  on  to  computer  disks  for  archiving.  They  wrote  and  submitted  key 
incident  reports  about  their  monitoring  strategies  according  to  the  guidelines  that  emerged  during  the  study  (sec 
Appendix).  The  key  incident  reports  emerged  as  a significant  data  source  which  illuminated  the  observations  and 
teacher  interviews  that  the  researcher  conducted.  A student  exit  survey  was  also  administered.  The  survey 
illuminated  important  characteristics  of  the  teachers’  monitoring  activities. 

Research  Findings  and  Discussion 

Analysis  of  the  key  incident  reports  showed  an  emphasis  on  tasks  requiring  students  to  reflect,  evaluate, 
and  appropriate  their  task  performance  decisions  during  computer-assisted  writing.  Three  categories  of  assignments 
emerged  in  the  data.  These  were  categorized  as  organizational  assignments,  revision  assignments,  and  MLA  citation 
and  documentation  assignments.  The  teachers  appeared  to  use  these  types  of  assignments  as  benchmarks  for 
assessing  students’  developmental  level  of  writing  skills.  Teachers’  monitoring  strategics  associated  with  the 
assignments  varied  from  student  to  student.  Multiple  assignments  were  accessible  on  the  network  server  for 
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students  to  complete  at  their  own  pace.  Students  routinely  engaged  in  the  computer-integrated  writing  activities 
independently  while  the  teacher  monitored  their  progress.  Teachers  used  a complex  arrangement  of  monitoring 
strategics  to  identify  students’  achievement  levels  in  order  to  facilitate  the  students’  incremental,  or  nearby,  skill 
development  along  a continuum  toward  instructional  goals.  Teachers  used  monitoring  strategies  to  remain  in  close 
proximity  to  the  students’  developmental  skill  needs  and  provided  context-appropriate  strategic  interventions 
documented  in  the  key  incident  reports. 

Seventeen  of  the  43  key  incident  reports  describe  the  use  of  open-ended  monitoring  strategies.  In  these  17 
reports,  the  teachers  reported  that  the  student  had  solved  his  or  her  own  problem  during  and/or  following 
monitoring.  In-depth  analysis  of  these  key  incident  reports,  including  the  student’s  before-and-after-instruction 
computer  files  and  transcribed  dialogue,  indicated  that  the  monitoring  experiences  had  resulted  in  the  students’ 
autonomous,  incremental  skill  development.  Teachers  provided  written  analysis  describing  the  consequences  of 
their  interactions  with  students,  drawing  from  evidence  in  the  tape-recorded  dialogue  and  observed  changes  in  the 
students’  computer-assisted  writing  performance.  A pattern  of  descriptive  monitoring  strategies  emerged  in  the  key 
incident  reports  where  teachers  reported  student  progress.  Descriptive  monitoring  strategies  are  characterized  by  a 
variety  of  open-ended  cues  and  interaction  tactics  used  by  teachers  to  elicit  reflective,  evaluative,  and  appropriativc 
responses  from  students. 

Another  nine  key  incident  reports  documented  teachers  use  of  directive  strategies  to  help  students  move 
forward  in  their  computer-assisted  writing  tasks.  These  reports  documented  that  the  students  had  mimicked  the 
teachers’  proposed  problem  solutions.  These  imitative  responses  also  constituted  incremental  skill  development  in 
the  students’  computer-assisted  writing  skills.  However,  the  skill  development  appeared  to  be  associated  with  a 
lower  level  of  learner  autonomy.  These  key  incidents  involved  teachers’  prescriptive  monitoring . Prescriptive 
monitoring  strategics  are  characterized  by  a variety  of  elosed-ended  comments  and  explicit  demonstrations  the 
teachers  used  to  pose  problem  solutions,  and  which  typically  elicited  imitative  responses  from  students. 

In  six  additional  key  incident  reports,  the  teachers  used  either  descriptive  or  prescriptive  monitoring 
strategies  that  did  not  result  in  students  exhibiting  incremental  skill  development  according  to  analysis  of  the 
students’  before  and  after  instruction  writing.  These  six  reports  showed  that  teachers  ineffectively  assessed  or  mis- 
diagnosed the  student’s  monitoring  needs  in  relation  to  the  assignment  tasks  and  their  respective  skill  level. 
Consequentially,  the  monitoring  strategics  teachers  used  in  these  cases  were  ineffective  for  the  given  learning 
context.  The  observation  notes,  unstructured  interviews,  key  incident  reports,  and  student  exit  survey  all  indicated 
that  a primary  function  of  teachers’  monitoring  was  to  engage  students  in  self-reflection,  self-evaluation,  and  self- 
appropriation activities  aimed  at  facilitating  their  ability  to  solve  their  own  computer-assisted  writing  problems.  The 
monitoring  strategies  teachers  used  represent  a complex  social-psychological  process  characterized  by  diagnostic 
monitoring  strategics,  followed  by  subsequent  instructional  branching  by  teachers.  Teachers  employed  a complex 
schema  of  monitoring  strategies  based  on  individual  student  response(s)  to  their  monitoring.  A dominant 
characteristic  of  the  instructional  process  that  facilitated  the  students’  incremental  skill  development  was  the  social 
interaction  between  teacher  and  student.  The  student  survey  indicated  that,  in  addition  to  the  social  value  of 
personal  interactions,  observing  interactions  between  chcrand  peer(s)  often  served  as  a learning  experience  for 
students.  Figure  1 shows  the  flow  of  interactions  observed  among  the  participants  of  the  study.  During  the 
interactions,  the  teacher  served  in  three  primary  roles  that  characterized  the  instructional  process:  co-evaluator, 
coach,  and  facilitator. 

Teacher  as  Co-Evaluator 

The  teacher  as  co-evaluator  put  emphasis  on  the  developmental  context  surrounding  students’  writing 
performance.  A student’s  individual  stage  of  development  as  a writer  emerged  as  a critical  factor  regulating  the 
teachers’  monitoring  activities.  Stage  of  development  refers  to  a given  student’s  level  of  skill  in  relationship  to  the 
computer-assisted  writing  goal  at  any  given  moment  during  the  observed  sessions.  Teachers  reportedly  selected 
their  monitoring  strategics  based  on  student’s  developmental  skill  needs.  The  teacher  acted  as  co-cvaluaf  r during 
collaborative  assessment  activities.  Peers  also  played  an  evaluation  role.  This  was  indicated  in  the  student  exit 
survey,  key  incident  reports,  and  observed  activities.  At  times,  teachers’  assigned  peer  evaluation  tasks  such  as 
online  peer  critique  exercises.  At  other  times  peer  evaluation  behaviors  arose  spontaneously  through  student 
initialed  interactions.  In  order  to  evaluate  students’  progress  during  CAI,  teachers  employed  collaborative 
assessment  stra  teg  i cs. 
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Figure  1.  Shows  a common  interaction  pattern  among  teacher,  students,  and  peers  observed  within  the 
computerized  learning  environment. 

The  reciprocal  social  interactions  between  the  teacher  and  his  or  her  students  during  computer-assisted 
writing  activities  fostered  contextual  learning  experiences.  The  teacher  used  collaborative  assessment  strategies  to 
engage  the  student  in  overtly  identifying  his  or  her  instructional  needs.  Through  the  collaborative  assessment 
experience,  the  teacher  and  student  became  aware  of  the  student’s  achievement  and/or  knowledge  level  as  it  related 
to  curricular  goals  and  objectives.  The  heightened  awareness  assisted  the  teacher  in  making  instructional  decisions 
and  appeared  to  prepared  the  student  for  learning.  The  following  monitoring  strategies  characterize  collaborative 
assessment: 

• Roaming , or  circulating  around  the  computer  lab  arena  while  students  work  on  their  assignments,  occurred  in 
conjunction  with  observing  to  check  students’  progress  toward  learning  objectives  and  curricular  goals. 

• Observing  appeared  vital  to  the  assessment  process,  i.e.  global  observing  of  the  whole  group  and  local 
individual  student  and  computer  screen-specific  observing  which  allowed  the  teacher  to  identify  each  student's 
progress  or  lack  of  progress. 

• Assessment  prompts  were  used  by  the  teacher  to  engage  students  in  dialogue  and  included  open-ended 
questions,  leading  questions,  and  exploratory  and  probing  cues,  which  were  designed  to  facilitate  dialogue 
leading  to  identifying  the  student’s  achievement  and/or  knowledge  level  and  to  clarify  any  problcni(s)  hindering 
student  progress. 

Teacher  as  Coach 

Another  teacher  role  was  that  of  “coach”;  teachers  used  this  word  to  describe  their  primary  instructional  role. 
Teachers  reportedly  used  coaching  strategics  to  provide  students’  with  guided  practice  experiences.  The  coaching 
process  found  in  the  data  involved  the  student  in  performance  of  a given  task  while  receiving  strategic  prompts  from 
the  teacher.  The  primary  distinction  between  coaching  and  evaluating  is  that  coaching  refers  to  instructional 
strategies  while  evaluating  refers  to  assessment  strategics.  In  actual  practice,  the  two  worked  hand-in-hand. 


2G5 


265 


The  teacher  determined  the  strategies  for  guiding  student  practice  of  a given  writing  task  based  on  the 
insights  generated  during  the  collaborative  assessment  process.  The  teachers  used  a variety  of  context-appropriate 
descriptive  and  prescriptive  monitoring  strategies  to  help  guide  the  students’  active  learning.  Through  reciprocal 
interactions  focused  on  a particular  problem  hindering  the  student’s  independent  performance  of  a given  task,  the 
teacher  provided  guidance  that  supported  the  student’s  incremental  skill  development.  The  strategies  listed  below 
illustrate  the  primary  descriptive  and  prescriptive  monitoring  strategies  found  in  the  data,  but  arc  not  meant  to  be  a 
comprehensive  listing.  Descriptive  monitoring  strategies  are  characterized  by  a variety  of  open-ended  cues  and 
interaction  tactics  used  by  the  teachers  to  elicit  reflective,  evaluative,  and  appropriate  responses  from  their 
students.  For  example; 

• Exploring  or  probing  a variety  of  alternative  methods  and/or  perspectives; 

• Mirroring  back  to  the  student  his  or  her  own  thinking  process  through  verbal  reiteration; 

• Negotiating  the  method  for  accomplishing  a task  or  the  meaning  of  a text; 

• Prompting  the  student,  to  think  in-depth  about  his  or  her  ideas  and  the  language  used  to  express  those  ideas 
within  an  argumentative  context;  and 

• Simulating  contexts  for  learning  that  resemble  situations  calling  for  autonomous  skill  performance,  i.e.,  counter- 
argumentation  techniques  or  Socratic  dialogue. 

Prescriptive  monitoring  strategies  are  characterized  by  a variety  of  elosed-ended  comments  and  explicit 
demonstrations  the  teachers  used  to  pose  problem-solutions,  and  which  typically  elicit  imitative  responses  from 
students.  For  example: 

• Explaining  or  elaborating  on  the  purposes,  beliefs,  procedures,  rules,  and/or  conceptual  substance  of  a 
computer-assisted  writing  task; 

• Goal-setting  to  make  explicit  the  characteristics  of  a task  and/or  the  subcomponent  of  the  task  that  acts  as  a 
benchmark  of  achievement; 

• Modeling  or  demonstrating  for  the  student  how  a given  task  may  be  accomplished;  and 

• Suggesting  specific  solutions  to  a given  writing  and/or  computer  usage  problem. 

Teacher  as  Facilitator 

A third  teacher  role  that  emerged  in  the  data  was  that  of  facilitator.  Although  the  teachers’  monitoring 
exhibited  many  characteristics  of  constructivist  approaches  to  learning,  with  an  ultimate  goal  of  writer  autonomy, 
teachers  did  use  directive  strategies  to  manage  the  instruction  process  on  an  individual  learner  level  and  a group 
level.  In  their  interviews,  teachers  described  the  diversity  of  spontaneous  social  behaviors  among  students.  The 
diverse  social  interactions  documented  in  the  researcher  observation  notes  also  indicated  variation  among  students. 
The  teachers,  through  their  global  comments  and  more  individualized  monitoring  strategies,  set  pacing  goals  that 
students’  continuously  modified  through  reciprocal  social  interactions.  This  phenomenon  indicated  a range  of 
individualized  writing  processes  among  the  180  student  participants  in  the  study.  Differences  in  writing  processes  of 
individual  student  writers  were  mentioned  in  the  technical  literature  as  well  (Berlin,  1994;  Hayes  & Flower,  1980; 
Tierney  & Shanahan,  1991).  While  the  characteristics  of  students’  individualized  writing  processes  was  not  the 
focus  of  this  research,  there  was  evidence  that  teachers  adjusted  their  monitoring  strategies  to  a range  of  student 
needs  for  facilitating  learning  throughout  an  instructional  session.  The  teacher  role  of  facilitator  is  characterized  by 
using  branching  strategics  to  promote  optimal  timing  and  sequencing  of  tasks  and  interactions  during  CAI. 

The  teachers  employed  instructional  branching  strategies  to  determine  the  optimal  sequencing  and  timing  of  the 
monitoring  cues  provided  to  students  during  computer-assisted  instruction.  Optimal  sequencing  of  a student’s 
assigned  tasks  was  determined  through  the  assessment-in-instruction  monitoring  pattern  unique  to  each  teacher- 
student  relationship.  The  teacher’s  optimal  timing  for  using  descriptive  and  prescriptive  monitoring  cues,  in 
relationship  to  assigned  tasks,  was  determined  through  collaborative  assessment  strategies.  Instructional  branching 
strategics  included  the  following: 
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• Effective  timing  of  appropriate  monitoring  cues  at  the  student’s  critical  point-of-nccd  in  order  to  facilitate 
incremental  skill  development 

• Management  of  multiple  interventions  in  a student’s  task  performance  activities  through  individuated,  small 
group,  and/or  global  interactions  that  may  be  teacher  or  student  initiated 

• Wait  time  following  a strategic  instructional  intervention  coupled  with  observing  the  student’s  consequential 
performance  level 

• Open-ended  assignment  agenda  supported  by  a computer  networked  archive  of  assignments  available  that 
facilitated  individualizing  the  instructional  process  based  on  the  student’s  progress  along  a continuum  toward 
curricular  goals  and  objectives 

The  data  showed  a relationship  between  the  time  a teacher  spent  engaging  a student  in  various  types  of 
monitoring  interventions  and  the  student’s  need  for  guidance  while  performing  a given  assignment.  Students  with 
less  autonomy  associated  with  a given  learning  goal  typically  received  more  prescriptive  monitoring.  Those  closer 
to  achieving  the  goal  received  more  descriptive  monitoring.  When  students  responded  well  to  prescriptive 
monitoring,  teachers  often  followed  up  with  descriptive  prompts.  The  decision  to  change  strategies  appeared  to 
stem  from  ongoing  collaborative  assessment.  Figure  2 depicts  the  instructional  process  the  researcher  calls  proximal 
instruction.  In  addition,  the  key  incident  reports  and  associated  transcribed  teacher-student  dialogues  illustrated  how 
a student’s  inability  to  respond  favorably  to  descriptive  monitoring  strategies  often  caused  the  teacher’s  to  branch  to 
prescriptive  monitoring  strategies.  This  phenomenon  indicates  the  ecletic  nature  of  teachers'  monitoring  strategics 
during  computer-assisted  writing  instruction.  Ultimately,  the  teachers'  instructional  goal  was  the  development  of 
students'  autonomous  writing  skills,  skills  characterized  by  self-monitoring. 


Proximal  Instruction  Model 


width  proportional  to  interaction  time 


Figure  2.  The  model  illustrates  the  relationship  between  the  time  teachers  spent  engaging  a given  student  in  various 
strategic  interventions  and  the  student's  apparent  need  for  guidance  while  performing  a given  task  in  pursuit  of  a 
learning  goal . The  arrows  represent  the  ongoing  collaborative  assessntent-in-instruction  and  the  branching 
strategies  used  by  the  participants. 
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The  students  engaged  in  collaborative  assessment  and  self-monitoring  experiences  guided  by  the  teachers. 
During  these  experiences,  the  teacher  typically  challenged  the  student  to  reflect  on,  evaluate,  and  appropriate 
solution(s)  for  a specific  problem  that  was  hindering  the  student’s  independent  task  performance.  As  a result,  the 
student  increasingly  managed  his  or  her  own  computer-assisted  writing  process  as  much  as  he  or  she  was  able, 
before  soliciting  intervention  from  the  teacher  or  a peer. 

This  research  identified  the  following  three  types  of  self-monitoring  skills  associated  with  computer-assisted  writing 
instruction: 

• Self- reflect  ion  involved  prompting  one’s  self  to  explore  and  articulate  one’s  ideas  through  writing. 

• Self  evaluation  involved  prompting  one’s  self  to  read  and  analyze  one’s  written  ideas. 

• Self-appropriation  involved  prompting  one’s  self  to  make  decisions  (including  revisions)  about  the  relevance 
and  applicability  of  one’s  ideas  in  a given  rhetorical  context. 

During  monitoring,  teachers  challenged  their  students  to  acquire  reflective,  evaluative,  and  appropriate 
skills  for  writing.  The  key  incident  reports  and  student  exit  survey  results  indicated  that  these  three  skills  were  the 
primary  focus  of  the  computer-assisted  composition  instruction  under  study.  The  researcher  interpreted  the 
teachers’  focus  on  helping  their  students  locate,  understand,  and  solve  their  task  performance  problems  as  a process 
of  facilitating  their  metacognitivc  skills  and/or  knowledge  related  to  writing  ability.  Metacognitive  knowledge 
“consists  primarily  of  knowledge  or  beliefs  about  what  factors  or  variables  [persons,  tasks,  and  strategies]  act  and 
interact  in  what  ways  to  affect  the  course  and  outcome  of  cognitive  enterprises”  (Flavell,  1979,  p.  907).  In  the 
substantive  area,  the  teachers’  monitoring  activities  focused  on  providing  students  with  metacognitive  strategy 
knowledge  through  interactive,  strategic  monitoring  experiences. 

Teachers’  monitoring  activities  resembled  characteristics  of  Vygotsky's  zone  of  proximal  development 
found  in  the  literature  (Bruner,  1986;  Dixon-Krauss,  1996;  Miller,  1993;  Vygotsky,  1978).  Therefore,  Vygotsky’s 
work  was  selected  as  a valid  developmental  theory  that  informed  the  development  of  the  principles  emerging  from 
this  research  project.  According  to  Bruner: 

In  contrast  with  learning  theories  and  developmental  theories  which  merely  describe  processes,  an 
instructional  theory  should  prescribe  the  optimal  arrangements  of  conditions  which  will  facilitate 
meaningful  school  learning.  An  instructional  theory  should  be  developed  alongside  learning  and 
development  theories,  and  its  principles  should  be  congruent  with  psychological  theories.  (Snelbcckcr, 
1985,  p.  419) 

Additional  characteristics  from  learning  theories  that  appeared  to  be  highly  relevant  to  the  teachers’ 
monitoring  process  were  found  in  literature  by  Feuerstein  and  Feuerstein  (1991),  i.e.,  the  concept  of  reciprocity 
between  teacher  and  student,  and  by  Dewey  (1939),  i.c.,  the  principle  of  educative  continuity.  Bruner’s  guidelines 
for  constructing  instruction  theory  were  used  to  develop  descriptions  of  the  four  principles  of  the  proximal 
instruction  process:  collaborative  assessment,  guided  practice,  instructional  branching,  and  self-monitoring  skills’ 
development.  These  principles  served  to  integrate  the  conceptual  categories  generated  from  the  analysis  of  the  data 
collected  for  this  study  and  provide  a basis  for  further  research  in  this  area  (Snelbcckcr,  1 985:  Miller,  1993). 

Teachers  monitoring  strategies  appeared  effective  in  identifying  the  point  at  which  students  were  unable  to 
perform  independently.  The  efficacy  of  teachers’  monitoring  practices  is  supported  by  the  following  statement  from 
the  technical  literature:  “Considering  choices  and  articulating  rationales  for  particular  composing  decisions  fosters 
metacognitive  awareness-a  hallmark  of  an  expert  writer”  according  to  David,  Gordon,  and  Pollard  (1995,  p.  530). 
Throughout  the  data  were  indicators  that  teachers  used  descriptive  monitoring  strategics  to  foster  students’  abilities 
to  make  their  own  writing  choices,  i.c.  self-evaluation  and  self-appropriation.  When  students  were  not  able  to 
independently  make  reasonable  writing  choices,  teachers  used  monitoring  strategies  to  assist  and/or  guide  students’ 
reflection  about  their  decision-making  process.  David,  Gordon,  and  PoPard  (1995)  point  out  that  “conscious  control 
of  one’s  own  writing  processes  [is]  a feature  of  expert  writers”  (p.  530).  The  students  involved  in  this  research  were 
novice  writers.  Participant  teachers  appeared  to  lead  students  through  monitoring  experiences  conducive  to 
development  of  their  own  self-monitoring  strategy  knowledge  and  skills. 


Characteristic  of  teacher  monitoring  was  the  goal  of  fostering  students’  autonomous  expression.  McCombs 
and  Whisler  (1989)  explain  autonomous  learning  from  a developmental  perspective.  “Psychologists  and  educators 
have  increasingly  recognized  that  learning  is  an  internally  mediated,  active,  generative,  and  constructive  process  of 
attending,  processing,  and  transforming  information  into  both  relatively  stable  and  dynamic  knowledge  structures” 
(p.  277).  These  researchers  point  out,  however,  that  learners  have  varying  abilities  to  assume  responsibility  for  their 
own  learning,  which  may  explain  why  teachers  were  found  branching  from  descriptive  to  prescriptive  monitoring 
strategies  based  on  individualized  interactions  with  students.  Furthermore,  McCombs  and  Whisler  identify  variables 
involved  in  the  developmental  process  of  becoming  an  autonomous  learner: 

To  become  autonomous  learners,  students  must  develop  various  cognitive  and  metacognitive  capabilities 
that  allow  them  to  actively  process  information,  attach  personal  meaning  to  learning  activities,  and  plan  and 
regulate  their  own  learning  activities.  At  the  same  time,  however,  they  must  also  develop  cognitive  and 
metacognitive  capabilities  for  controlling  and  regulating  affect  and  motivation.  The  development  of  these 
latter  capabilities  depends  on  the  development  of  self-system  structures  (schemas  for  organizing  self- 
knowledge,  beliefs,  values,  and  goals)  and  self-system  processes  (particularly  self-awareness  and  self- 
evaluation  processes  related  to  personal  competence  and  control),  (p.  278) 

A predominant  theme,  in  the  key  incident  reports  written  by  the  teachers  and  in  their  interviews,  was  the 
focus  on  transferring  the  monitoring  responsibility  from  teacher  to  student  over  time.  A primary  characteristic  of  the 
instructional  process  that  emerged  from  analysis  of  the  CAT  goals  identified  in  the  key  incident  reports  was  the 
dimension  of  writer  autonomy,  facilitating  students'  ability  to  fully  engaging  in  discourse  communities.  This 
characteristic  of  the  instruction  is  in  keeping  with  the  goals  within  the  field  of  composition  studies  identified  by 
many  (Balcster,  Halasck  & Peterson,  1992;  Hawishcr,  1990;  Probst,  1990).  David  et  al.  (1995)  explain:  “But  to 
become  full  participants  in  such  [academic  discourse]  communities,  students  need  a sense  of  themselves  as  writers 
which  can  best  be  developed  in  a writing  course  grounded  in  the  assumptions  [that]  the  development  of  writing 
ability  and  metacognitive  awareness  is  the  primary  objective  of  a writing  course.  . . . The  students’  writing  is  the 
privileged  text  in  a writing  course.  . . .The  subject  of  a writing  course  is  writing”  (p.  525,  530).  The  goals  of  the 
curriculum  under  study  aligned  with  these  three  assumptions  about  composition  studies. 

The  collective  data  showed  that  the  computer  makes  the  cognitive  processes  associated  with  students’ 
writing  more  concrete  and  explicit  for  both  the  teacher  and  the  student.  The  computer’s  capacity  to  make  visible 
previously  undisclosed  phenomena,  in  general,  was  recognized  by  Zuboff  (1988).  Zuboff  coined  the  term 
“informating”  to  describe  the  computer-bound  phenomena:  “The  consequences  of  the  technology’s  informating 
capacity  arc  often  regarded  as  unintended.  Its  effects  are  not  planned,  and  the  potential  that  it  lays  open  remains 
relatively  unexploited.  Because  the  informating  process  is  poorly  defined,  it  often  evades  the  conventional 
categories  of  description”  (p.  1 1). 

The  computer  played  a significant  role  in  bringing  to  the  foreground  of  the  learning  environment  the 
students'  actual  writing  process.  The  computer  was  used  as  a primary  tool  supporting  an  instruction  approach  that 
allowed  the  student  to  engage  in  process  writing  activities  while  the  teacher  observed  and  monitored  those  activities. 
Assignments  drawing  on  the  unique  features  of  the  computer  were  used  during  the  sessions  under  study. 
Assignments  used  a number  of  word  processing  features  to  support  student  writing  activities.  These  included  the 
multi-file  function  to  display  an  assignment  file  and  a student  workspace  on  one  screen.  Assignment  files  varied 
from  types  of  models  (i.c.  prospectus,  thesis  statements,  works  cited  examples)  to  exercises  (i.e.  sentence  revision 
exercises,  peer  critique  directions,  question  and  answer  prompts).  Students  were  required  to  use  columns  to  align 
argument  and  counter-argument  points  and  the  outlining  function  to  their  organize  writing.  They  learned  to  use  the 
revision  tracking  function  to  write  and  evaluate  peer  comments.  They  used  the  cut,  paste,  delete  and  insert  functions 
to  facilitate  macrostructure  revision  and  the  spell  checker,  thesaurus,  grammar  checker,  and  readability  index  to 
facilitate  microstructurc  revisions.  During  all  these  events,  the  teacher  monitored  the  students'  computer-assisted 
process  and  actually  became  involved  in  the  process,  while  non-comDutcrizcd  process  writing  workshops  are  limited 
to  analysis  of  writing  products,  i.c.,  drafts. 

In  the  computerized  environment,  student  engagement  in  writing  activities  freed  the  teacher  to  individually 
monitor  and  observe  student  writing  progress.  In  addition,  the  emphasis  on  writing  activity  allowed  the  teacher  to 
authentically  compare  students’  achievement  levels  within  a peer  group.  Ultimately,  the  computer  facilitates  the 
instructional  process  by  challenging  students  to  work  independently  as  far  as  they  are  able  on  a given  computer- 
integrated  writing  task.  In  this  regard,  the  computerized  setting  under  study  facilitated  proximal  instruction  aimed  at 
fostering  students’  independent  writing  skills.  In  sum,  the  computer-assisted  composition  instruction  provided  a 


way  for  teachers  to  become  closer  to  their  students'  skill  development  processes.  At  the  same  time,  it  brought  to  the 
foreground  hidden  characteristics  of  the  writing  process.  These  included  reading  comprehension,  text  analysis, 
vocabulary  building,  information  researching,  critical  thinking,  creative  thinking,  and  computer  literacy  skills.  Some 
characteristics  of  the  proximal  instruction  process,  as  described  in  this  report,  deserve  further  research. 
Recommendations  for  further  research  include: 

• The  data  suggests  that  prescriptive  monitoring  strategies  foster  skill  development  levels  that  are  antecedent  to 
those  fostered  by  descriptive  monitoring  strategies;  however,  the  critical  junctures  and  the  effects  of  the  various 
monitoring  strategies  in  relationship  to  students’  developmental  writing  skill  level  needs  further  research. 

• The  association  between  teachers’  monitoring  cues  and  the  learner's  motivational  propensity,  extrinsic  or 
intrinsic,  was  not  verifiable  by  the  data  collected  during  this  research  project.  Further  research  is  need  to 
uncover  the  nature  of  the  motivational  factors  influencing  writing  skill  development  and  self-monitoring. 

• The  effects  of  a teacher's  monitoring  in  relationship  to  the  development  of  a student’s  mctacognitivc  monitoring 
skills  for  writing  needs  investigation. 

This  study  describes  a complex  monitoring  process  used  by  teachers  as  they  individualized  their  instructional 
approach  in  a computerized  learning  environment.  Findings  from  this  study  may  inform  instructional  system 
designs  to  assist  teachers  and  student  in  collecting  and  monitoring  the  proliferation  of  data  that  emerges  when 
students  take  an  active  learning  role  within  the  classroom.  The  next  study  addresses  critical  questions  associated 
with  designing  computer  systems  for  performance-based  assessment. 

Investigating  Performance-Based  Assessment  Management  Systems 

The  widespread  concern  that  students  in  the  American  educational  system  are  failing  to  learn  critical 
thinking,  problem-solving,  and  reasoning  skills,  and  lack  the  ability  to  transfer  these  skills  to  authentic  tasks  has 
become  a focus  for  our  national  educational  reform  agenda.  “As  it  is  now,  students  are  rarely  taught  to  learn  how  to 
learn  - that  is,  how  to  ‘manage’  knowledge  so  as  to  effectively  store  and  retrieve  it  for  thoughtful,  flexible  use-nor 
are  they  assessed  in  such  a way  as  to  test  their  ability  to  manage  available  resources”  (Wiggins,  1993,  p.  84). 
“Learning  to  learn”  incorporates  the  use  of  information  and  prior  experiences  in  creative  and  imaginative  ways, 
preferably  in  a risk-free  learning  environment  that  supports  thinking.  According  to  Wiggins  (1993),  this  form  of 
thinking  is  not  widely  encouraged;  therefore,  it  is  no  surprise  that  the  development  of  assessment  processes  to 
evaluate  this  form  of  thinking  has  not  been  a high  priority.  “Students  learn  to  fear  admitting  ignorance  or  being 
creative.  Questionable  or  imaginative  responses,  rather  than  being  valued  by  students  as  the  building  blocks  of 
thoughtful  understanding,  arc  viewed  nervously  as  potential  mistakes”  (Wiggins,  1993,  p.  73).  In  order  to  deal  with 
these  concerns,  educators  and  psychologists  have  been  researching  various  methods  of  improving  both  student 
learning  and  the  assessment  of  student  achievement.  “Educational  assessment  is  in  a process  of  invention.  Old 
models  arc  being  seriously  questioned;  new  models  are  in  development”  (Herman,  1992,  p.131).  This  view  is 
supported  by  Wiggins  (1993)  as  he  cites  Norman  Frederikscn,  a senior  Educatior  .1  Testing  Service  (ETS) 
researcher  who  states:  “Most  of  the  important  problems  one  faces  in  real  life  are  ill  stmetured,  as  are  all  the  really 
important  social,  political,  and  scientific  problems  in  the  world  today.  But  ill-structurcd  problems  arc  not  found  in 
standardized  achievement  tests...  We  need  a much  broader  conception  of  what  a test  is  if  we  are  to  use  test 
information  in  improving  educat  onal  outcomes”  (p.  4). 

In  the  wake  of  concerns  about  standardized  measures,  individual  teachers,  districts,  and  states  arc 
developing  new  kinds  of  assessment  measures  based  on  performance  outcome  criteria.  In  order  to  gain  the 
necessary  assessment  data  that  can  provide  teachers  with  accurate  and  useful  information  concerning  student 
performance,  teachers  arc  turning  away  from  traditional  summative  evaluation  methods  toward  formative  methods 
of  assessment.  These  methods  of  assessment  are  known  as  authentic  assessment,  alternative  assessment,  and 
performance-based  assessment.  These  terms  arc  often  used  interchangeably  with  each  other;  however,  they  arc  not 
the  same.  According  to  Marzano,  Pickering,  & McTighe  (1993,  p.  13),  authentic  assessment,  popularized  by  Grant 
Wiggins  (1989),  conveys  the  idea  that  assessment  should  engage  students  in  applying  knowledge  and  skills  in  the 
same  way  they  are  used  in  the  “real  world”  outside  of  school.  Alternative  assessment  applies  to  the  variety  of 
assessments  that  differ  from  the  multiple-choice,  timed,  one-shot  approaches  characterized  by  most  traditional 
standardized  and  classroom  assessments.  Finally,  performance-based  assessment,  according  to  Marzano,  Pickering, 
# McTighe  (1993,  p.13),  is  a broad  term  encompassing  many  of  the  characteristics  of  both  authentic  assessment  and 
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alternative  assessment.  The  Office  of  Technology  Assessment  (OTA)  defines  performance-based  assessment  of  the 
U.S.  Congress  (1992)  as  “testing  that  requires  a student  to  create  an  answer  or  a product  that  demonstrates  his  or  her 
knowledge  or  skills”.  Performance-hascd  assessment  requires  students  to  ‘‘do”  something  as  opposed  to  taking  an 
objective  paper-pencil  test.  It  zeros  in  on  what  a student’s  strengths  arc  and  what  a student  needs  to  learn,  and 
requires  students  to  actively  accomplish  complex  and  significant  tasks  based  on  experiential  learning  and  relevant 
skills  to  solve  realistic  or  authentic  problems.  “...Assessment(s)  in  context  have  ‘ecological  validity' — that  is, 
students  perform  as  they  will  have  to  in  life”  (OERI,  1990.  p.  1).  This  assessment  process  is  designed  to  present  a 
broader,  more  genuine  picture  of  student  learning. 

Performance  assessment,  as  defined  by  Mitchell  (1992,  p.  20),  is  “a  collection  of  ways  to  provide  accurate 
information  about  what  students  know  and  arc  able  to  do  or  about  the  quality  of  educational  programs.” 
Assessment,  therefore,  should  be  seen  as  a motivational  component  to  give  both  the  teacher  and  the  student  the 
necessary  feedback  to  enhance  the  learning  process.  Hence,  “Assessment. ..becomes  a part  of  the  instruction,  even 
when  the  results  of  assessment  are  also  used  for  accountability”  (Mitchell,  1992,  p.  21).  When  assessment  and 
instruction  arc  viewed  as  a simultaneous  process,  as  in  performance-based  assessment,  the  learner  is  provided  with  a 
clear  understanding  of  the  criteria  necessary  to  confidently  complete  his/her  task  successfully.  The  establishment  of 
an  accepted  set  of  criteria  for  the  measurement  of  a performance-based  activity  or  project  requires  standard 
procedures  and/or  methods  in  data  gathering.  The  use  of  uniform  procedures  and/or  methods  for  data  gathering  are 
not  meant  to  place  constraints  on  teachers’  qualitative  judgment  of  the  content  to  be  addressed  in  the  assessment 
process.  It  is  meant  to  help  in  identifying  the  basis  of  measuring  or  judging  the  process  as  well  as  the  product  of  a 
student’s  activity.  “An  emphasis  on  content  without  process  means  honoring  the  ends,  not  the  means,  of  education” 
(Costa  & Liebmann,  1995,  p.  23). 

Classroom  teachers  find  themselves  in  an  instructional  environment  that  demands  that  they  face  myriad 
changes  in  content,  context,  and  delivery  of  information  to  students.  These  changes  have  affected  not  only  the 
classroom  instructional  environment,  but  also  have  impacted  management  issues  dealing  with  the  collection  of 
student  assessment  data.  The  ability  to  access  assessment  data  from  multiple  sources  in  a time  efficient  manner  and 
the  ability  to  interpret,  synthesize,  and  apply  the  information  are  rapidly  becoming  major  challenges  for  the 
classroom  teacher.  The  need  to  manage  student  assessment  data  is  critical  for  effective  educational  diagnosis  and  is 
an  important  component  in  today’s  climate  of  academic  accountability.  The  data  gathering  process  for  performance 
assessment  must  be  consistent  and  systematic  in  order  to  acquire  reliable  and  valid  data  concerning  student 
performance.  Performance-based  assessment  data  is  collected  through  the  use  of  anecdotal  records,  journals, 
rubrics,  portfolios,  projects,  presentations  and/or  interviews.  This  form  of  data  gathering  is  complex,  potentially 
massive,  time  consuming,  costly,  and  is  inherently  difficult  to  manage  with  a class  of  twenty-five  or  more  students. 
Therefore,  if  performance-assessment  is  to  succeed,  support  for  the  classroom  teachers  to  effectively  collect  and 
manage  performance-based  assessment  is  a strategic  management  issue  facing  the  educational  system. 

An  area  that  offers  great  potential  for  meeting  the  management  problems  created  by  the  vast  amounts  of 
complex  data  is  technology.  However,  in  the  search  to  find  effective  solutions,  educators  must  be  weary  of  vendors 
touting  packaged  solutions  too  quickly.  Just  as  performance-based  assessment  is  unique  and  complex,  so  is  the 
process  of  collecting  and  managing  the  data.  Through  the  use  of  technologies  such  as  computers,  computer  software, 
scanners,  and  handheld  PCs,  the  intricate  process  of  collecting,  managing  and  interpreting  performancc-bascd 
assessment  can  be  addressed.  Many  software  designers  have  recognized  this  educational  market  and  have  eagerly 
entered  the  educational  field  with  various  assessment  management  software  programs.  The  majority  of  these 
programs  are  designed  using  database  and/or  multimedia  programs.  Both  applications  have  the  capability  of  solving 
the  management  issm  s involved  in  performance-based  assessment.  However,  a major  concern  surrounding  these 
management  programs  is  many  designers  have  only  a surface  level  understanding  of  the  intricacies  involved  in 
pcrformancc-based  assessment  as  a component  in  the  total  schema  of  a classroom  environment. 

The  classroom  environment  is  unique  to  each  class  of  students,  school,  and  teacher  therefore  a 
performance-based  assessment  management  system  aimed  at  meeting  classroom  assessment  management  needs 
should  include  components  geared  towards  the  classroom.  The  purpose  of  this  study  is  to  examine  the  impact  of 
technology  on  performance-based  assessment  and  to  determine  ways  to  infuse  it  into  the  educational  process  in 
order  to  solve  the  management  issues  facing  the  classroom  teacher.  This  study  is  intended  for  designers, 
researchers,  administrators,  technology  coordinators,  and/or  classroom  teachers  interested  in  understanding  the 
impact  of  technology  on  the  management  of  performance-based  assessment. 

The  research  questions  addressed  in  this  study  were  1.)  How  can  technologies  such  as  videos,  computers, 
scanners,  and  others  provide  standardization  of  the  assessment  process,  ensure  timc-cffcctivencss,  and  identify  cost 
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efficient  methods  for  the  facilitation  of  classroom  management  of  pcrformancc-bascd  assessment?  2.)  How  can 
technologies  improve  reliability  and  validity  in  the  performance-based  assessment  process?  3.)  What  type  of 
guidelines  can  be  developed  in  order  to:  1)  provide  standardization  of  the  assessment  process,  ensure  time- 

effectiveness,  and  identify  cost  efficient  methods  for  the  facilitation  of  classroom  management  of  performance- 
based  assessment?  2)  improve  reliability  and  validity  in  the  pcrformancc-bascd  assessment  process? 

Research  Method 

The  methodology  used  in  this  qualitative  research  study  was  inductive  analysis.  Inductive  analysis, 
according  to  Patton  (1990),  “...means  that  the  patterns,  themes,  and  categories  of  analysis  come  from  the  data;  they 
emerge  out  of  the  data  rather  than  being  imposed  on  them  prior  to  data  collection  and  analysis1’  (p.  390).  This 
research  study  was  conducted  in  four  stages  to  fully  identify  the  patterns,  themes,  and  categories  of  analysis. 

First  stage  included  an  extensive  literature  review  and  document  analysis  that  provided  the  framework  for 
three  sensitizing  concepts  to  emerge.  A more  extensive  research  of  the  three  sensitizing  concepts  provided 
indigenous  typologies  to  emerge.  The  characteristics  or  attributes  that  comprise  the  indigenous  typologies  of  the 
categories  within  the  identified  sensitizing  concepts  were  then  used  as  the  criteria  in  a rubric.  This  rubric  was  then 
used  in  Stage  Two  and  Three  as  an  analysis  tool.  It  was  used  to  analyze  five  assessment  software  programs  as  well 
as  provide  a guide  in  the  conversational  interviews  that  were  conducted.  The  interviewees  included  designers  of 
software  programs,  consultants  representing  the  five  performance-based  assessment  software  programs,  and  teachers 
who  had  either  used  or  were  currently  using  one  of  the  software  programs  in  the  study.  The  data  from  Stages  One, 
Two,  and  Three  provided  the  foundation  necessary  for  Stage  Four,  determining  the  findings  of  the  study  and 
presenting  the  results  of  the  study,  a set  of  guidelines  for  the  management  of  performance-based  assessment. 

Research  Findings  and  Discussion 

The  results  of  this  study  are  proposed  guidelines  for  the  management  of  performance-based  assessment. 
The  design  of  the  proposed  guidelines  is  the  result  of  a multi-level  research  process.  This  multi-level  research 
process  surfaced  the  information  needed  to  develop  comprehensive  guidelines  that  utilize  technology  to  effectively 
manage  performance-based  assessment  and  thus  positively  impact  the  work  of  the  classroom  teacher.  The  guidelines 
developed  from  this  research  consist  of  four  basic  components:  (a)  standardization  of  process;  (b)  time 

commitment;  (c)  cost;  and  (d)  operational  features. 

Acknowledging  and  documenting  the  multi-dimensional  learning,  thinking  and  performat.ee  capabilities  of 
all  students  engaged  in  performance-based  tasks  requires  a practical  and  flexible  assessment  management  system. 
The  ability  to  systematically  organize  large  amounts  of  complex  assessment  data  is  critical  for  the  success  of 
performance-based  assessment.  Therefore,  the  following  guidelines  have  been  developed  to  assist 
educators/administrators,  technology  coordinators,  software  designers,  and/or  researcher  involved  in  selection  or 
design  of  performance-based  assessment  management  programs.  The  guidelines  bring  together  the  various 
components  of  performance-based  assessment  that  range  from  obvious  and  commonplace  issues  to  those  that  arc 
somewhat  technical  and  specialized.  In  addition,  the  guidelines  provide  a framework  from  which  to  determine  the 
worth  of  an  existing  performance-based  assessment  system/program  or  to  use  as  a guide  in  the  development  of  a 
new  program  for  effectively  managing  performance-based  assessment.  A graphic  of  the  guidelines  is  located  in 
Table  1. 

The  guidelines  for  the  management  of  performance-based  assessment  consist  of  four  basic  components:  (a) 
standardization  of  process;  (b)  time  commitment:  (c)  cost;  and  (d)  operational  features.  The  first  and  most  critical 
of  these  components  is  standardization  of  process.  A performance-based  assessment  management  program  must 
have  a consistent  system  that  ensures  validity  and  reliability  in  assessment  situations  as  well  as  a historical  or 
benchmark  representation  of  student  progress.  In  the  proposed  guidelines,  the  issues  of  accuracy,  multiple  samples, 
data  gathering  flexibility  and  consistency  arc  addressed  to  ensure  a valid  assessment  process  that  reflects  what  a 
student  knows  or  can  do.  The  issues  of  data  entry,  replication,  historical  review,  and  training  arc  addressed  to  entire 
the  reliability  of  an  assessment  process.  Reliability  involves  establishing  a clear  understanding  of  the  criteria  and 
standards  to  be  used  in  the  assessment  process  through  the  recording  of  assessment  data  using  a rubric,  anecdotal 
and  journal  notations,  video  clips,  photographs,  hand  drawn  pictures,  or  written  reports.  The  ability  to  record 
assessment  data  using  a variety  of  methods  enables  a deeper  understanding  of  students’  progress;  it  also  gives  a 
clearer  picture  of  students’  current,  as  well  as  historical,  academic,  and  social  progress.  The  ability  to  review 
student  progress  over  time  is  a powerful  tool  in  the  academic  and  social  development  of  the  whole  child.  Capturing 
student  progress  through  careful  data  gathering  over  time  enables  teachers  to  gain  insight  into  the  progress  and 
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development  of  students  who,  for  example,  may  he  struggling,  may  have  recently  moved  into  the  school,  or  who 
may  have  exhibited  changes  in  learning  habits. 

While  the  capability  of  multiple  data  entries  allows  for  a more  comprehensive  picture  of  student  progress,  it 
can  also  negatively  impact  consistent  replication  of  student  assessment  results.  Preparing  teachers  to  become  more 
accurate  and  consistent  in  gathering  and  documenting  assessment  data  is  critical  to  the  success  of  any  performance- 
based  assessment  program.  The  need  for  accurate  and  consistent  replication  of  assessment  results  drives  major 
issues  in  developing  a well-designed  training  program.  A carefully  and  systematically  designed  training  program 
supports  teachers  in  the  implementation  of  performance-based  assessment  and  helps  to  ensure  that  the  same  rating 
can  be  replicated  for  a given  student  by  another  teacher  at  another  time,  thus  supporting  the  reliability  of  the 
assessment  program. 

This  systematic  training  can,  in  theory,  increase  teachers’  confidence  in  their  ability  to  utilize  performance 
based  assessment  by  providing  both  the  tools  and  the  skills  to  use  up-to-date  and  accurate  data  in  an  effective  way. 
As  confidence  grows,  utilization  expands  and  along  with  it  follows  efficiency,  actually  giving  teachers  time  to 
reflect  on  the  delivery  of  information,  on  content,  or  on  a combination  of  the  two.  Engaging  in  an  effective 
reflection  process  allows  teachers  to  more  clearly  identify  and  modify  educational  goals  in  a timely  manner,  making 
adjustments  to  curriculum  as  the  need  becomes  apparent.  For  years  students  have  been  given  standardized  tests,  but 
the  results  were  not  returned  to  the  classroom  teacher  until  the  following  school  year,  too  late  to  effect  any  changes 
which  would  positively  impact  student  learning.  Having  the  skills  to  use  ready  access  to  assessment  information 
about  the  process,  progress,  and  final  product  of  students'  work  should  greatly  enhance  teachers1  confidence  in  their 
ability  to  make  meaningful  curricular  adjustments  and  in  their  overall  view  of  themselves  as  effective  professionals. 

The  guidelines  address  time  commitment  on  two  levels:  the  time  required  to  learn  disciplined  use  of  an 
assessment  program,  a potential  perceived  negative,  and  the  time  saved  and  accuracy  gained  by  use  of  an  assessment 
program,  a clear  positive.  The  first  issue  is  addressed  by  devising  a training  plan  with  an  appropriately  designed 
learning  curve  so  those  teachers  are  challenged  without  being  overwhelmed.  The  remaining,  and  most  important, 
issue  involves  how  teachers  recognize  the  relevance  and  usefulness  of  a truly  functional  assessment  management 
program.  As  teachers’  data  gathering  and  documentation  skill  levels  increase,  the  amount  of  time  required  to  collect 
and  manage  extensive  amounts  of  assessment  data  will  decline.  They  will  begin  to  realize  and  welcome  the  power  of 
systematically  analyzing  and  utilizing  assessment  data  for  diagnostic  purposes  to  improve  the  learning  process,  as 
well  as  to  enhance  accountability.  Gradually,  teachers  will  begin  to  understand  the  time  saving  value  and  the 
improved  quality  of  information  input  and  output  with  an  assessment  program  designed  with  the  use  of  the  proposed 
guidelines. 

Conceptually,  standardization  of  process  should  be  viewed  as  the  critical  component  in  the  guidelines. 
Practically  however,  the  critical  component  is  often  the  initial  and  ongoing  cost  of  the  program.  The  proposed 
guidelines  point  out  the  need  for  a clearly  defined  cost  structure  that  addresses  the  needs  of  single  users  and  small  or 
large  school  districts.  Incorporating  a flexible  yet  clearly  defined  cost  structure  provides  single  users  as  well  as 
district  users  the  option  of  customizing  or  phasing  in  a program.  In  this  way,  districts  can  purchase  only  what  they 
need  or  can  afford,  thus  optimizing  the  use  of  available  monies. 

Finally,  the  operational  features  of  the  guidelines  arc  both  practical  and  futuristic.  The  idea  of  connecting 
the  schools  electronically  to  the  rest  of  the  world  opens  up  new  educational  frontiers  for  students  to  enter  and  learn. 
Connectivity  offers  the  ability  to  learn  in  new  and  exciting  ways  with  learning  partners  never  before  available  to 
st  uden  ts  i n grades  K- 1 2. 

Along  with  new  learning  opportunities,  connectivity  presents  major  challenges  for  the  classroom  teacher  in 
effectively  assessing  learning  in  this  new  electronic  learning  environment.  These  assessment  challenges  can  be  met 
by  first  gaining  a thorough  knowledge  of  performance-based  assessment.  Initially  designing  assessments 
instruments  to  support  and  complement  tasks  performed  in  an  electronic  environment  for  students  to  achieve  their 
goals  will  greatly  enhance  both  learning  and  the  assessment  of  learning.  The  systematic  and  organized  integration 
of  the  components  of  connectivity  into  the  assessment  process  will  lay  the  groundwork  for  the  next  logical  step, 
which  is  the  connectivity  of  our  schools  to  the  vast  possibilities  of  learning  available  to  them  through  e-mail,  bulletin 
boards,  home  pages,  and  the  WWW.  It  also  opens  up  a new  vehicle  for  communication  with  parents,  teachers, 
administrators,  colleagues,  community  libraries,  local  government,  etc.  that  will  be  of  great  practical  benefit. 

Theoretically,  connectivity  has  the  potential  to  both  narrow  and  broaden  communication  between  the  home 
and  school.  It  can  narrow  communication  if  parents  and  teachers  resort  to  strictly  “on-line”  communication.  The 
negative  impacts  of  using  only  this  form  of  communication  arc  that  1)  many  individuals  find  it  difficult  to  fully 
express  themselves  in  written  form,  2)  the  loss  of  face-to-face  contact  can  negatively  affect  communication  due  to 
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Ihc  inability  to  interpret  subtle  messages  expressed  through  body  language,  and  3)  if  the  conversation  is  not  “live- 
on-line",  the  spontaneous  reactions  or  attention  may  be  lost,  thus  limiting  communication.  Connectivity  can  broaden 
communication  through  the  ready  access  available  to  both  parents  and  teachers  “on-line”. 

Communication  between  teachers  and  parents  is  difficult  during  the  school  and  workday.  E-mail  allows 
interested  parties  to  send  messages  at  any  time  that  can  be  received  and  answered  at  a convenient  time.  Teachers 
can  electronically  send  the  same  message  to  all  students  simultaneously  or  can  write  short  notes  to  parents 
concerning  the  successes  as  well  as  difficulties  of  individual  children  to  keep  parents  abreast  of  classroom  activities 
and  conduct.  Finally,  teachers  or  parents  do  not  have  to  be  in  a specific  location  to  communicate  with  each  other.  If 
a parent  travels,  he  or  she  can  log  on  to  receive  updates  about  their  children’s  progress  and  respond  from  anywhere 
in  the  world.  These  positive  and  negative  scenarios  must  be  dead  with  as  a natural  outgrowth  of  change.  The  pace 
of  change  is  increasing  at  an  exponential  rate  resulting  in  boundless  opportunities  for  those  who  arc  prepared. 
Education  should  provide  that  preparation  by  forging  ahead  to  set  trends  in  learning  that  extend  beyond  the  walls  of 
the  traditional  classroom. 

The  assessment  process  is  an  integral  part  of  learning  and  teaching.  Performance-based  tasks  allow 
students  to  show  the  process  as  well  as  a final  product  of  a learning  situation.  In  this  form  of  learning  the 
assessment  is  no  longer  a mystery,  but  an  understood  standard  by  which  to  measure  student  work  in  progress. 
Demystifying  the  methods  of  assessment  enables  the  learning  process  to  take  on  new  meaning.  One  where  risk- 
taking  is  encouraged  in  order  to  extend  critical  thinking  and  reasoning  skills  beyond  the  expectation  of  “making  a 
grade”  based  on  recall  and  identification  to  an  active,  engaged  performance  or  demonstration  of  ability.  The 
methods  involved  in  performance-based  assessment  arc  seen  as  tools  in  the  learning  process  for  both  the  student  and 
the  teacher  in  order  to  improve  and  encourage  the  process  as  we1!  as  the  end  product  of  learning. 

Performance-based  assessment  is  not  a panacea;  in  fact,  it  holds  many  challenges  for  the  educational 
community.  If  done  in  its  true  form,  performance-based  assessment  generates  large  amounts  of  complex  assessment 
data  that  require  careful  and  systematic  organization  in  order  to  assist  with  and  measure  student  learning.  The 
development  of  a performance-based  assessment  management  program  rooted  in  the  proposed  guidelines  can  be  of 
great  benefit  in  solving  many  of  the  challenges  faced  by  the  classroom  teacher  as  well  as  in  the  overall  learning 
process. 

Students  must  be  prepared  to  compete  in  the  workforce  of  the  future,  adapt  to  diverse  learning  contexts,  and 
collaborate  with  others  on  a variety  of  local  and  global  levels.  The  ability  to  assess  their  own  work  in  progress,  as 
well  as  their  own  final  product,  will  be  an  essential  asset  for  the  student  of  today  and  the  worker  of  tomorrow  to 
possess.  “Workforce  know-how  will  be  part  of  the  new  World  Class  Standards.  However,  defining  competencies 
and  a foundation  is  not  enough.  Schools  must  teach  them.  Students  must  learn  them.  And,  they  should  be  assessed 
as  part  of  the  AMERICA  2000  agenda”  (SCANS  report  for  America  2000,  1995,  p.  12). 

Educational  Implications  from  the  Studies 

In  school  districts  implementing  technology  into  the  classroom,  teachers  find  themselves  faced  with 
demands  requiring  changes  in  content,  context,  and  methods  of  instruction.  The  research  investigating  teacher’s 
monitoring  activities  a performance-based  learning  environment  found  that  when  learners  become  actively  engaged 
in  learning  tasks  a rich  variety  of  learner  characteristics  and  variables  come  to  the  foreground  of  the  instructional 
arena.  Learner  characteristics  emerge  that  arc  not  readily  apparent  in  traditional  chalk-talk-textbook  instruction 
where  students  play  a more  passive,  receptive  role.  Because  learning  becomes  more  individualized,  students  must 
learn  to  mctacognitivcly  manage  and  direct  their  learning  activities  as  much  as  they  arc  able  in  order  to  be  active 
participants  in  authentic  learning  tasks.  The  research  suggests  ways  in  which  performance-based  learning  events  can 
be  structured  so  that  teachers,  students,  and  technology  systems  effectively  assist  the  development  of  self-regulatory 
learning  skills. 

While  authentic,  performance-based  learning  tasks  arc  associated  with  technology  implementation,  the 
changes  that  implementation  cause  in  the  classroom  challenge  the  teacher  with  new  classroom  dynamics  to  manage. 
Proximal  instruction  describes  instructional  strategies  for  managing  the  social  interactions  within  a computerized 
learning  environment.  Changes  brought  about  by  technology  not  only  affect  instructional  methods,  but  also  impact 
management  issues  related  to  student  assessment  of  learning.  The  ability  to  archive,  synthesize,  interpret,  and  apply 
relevant  assessment  data  from  performance-based  learning  events  in  a time  efficient  manner  is  also  a critical  concern 
associated  with  implementation  of  technology  in  the  classroom.  The  proposed  guidelines  derived  from  research 
bring  together  the  various  components  of  pcrformancc-hascd  assessment  that  range  from  the  commonplace  to 


technical  issues  influencing  the  design,  implementation,  and  utilization  of  performance-based  assessment 
management  systems  that  can  greatly  improve  the  overall  learning  process. 

Performance- based  learning  is  a complex,  individualized  educational  approach  that  requires  different 
instructional  strategies  and  technological  tools  to  support  and  properly  implement.  In  this  paper,  the  authors  have 
presented  research  finding  and  draw  the  following  conclusions. 

• Technology  systems  need  to  be  designed  in  congruence  with  teachers  instructional  approach  in  order  to 
facilitate  student’s  proximal  development  toward  an  instructional  goal. 

• The  monitoring  strategies  and  the  system  design  guidelines  presented  :n  this  paper  need  be  contextualized  to  fit 
different  instructional  goals  in  different  content  areas. 

• The  descriptions  of  the  proximal  instruction  process  and  the  performance  assessment  system  guidelines  may 
provide  interested  individuals  with  insight  into  factors  to  consider  when  making  instructional  design  decisions. 

The  research  studies  identify  the  need  for  technological  solutions  to  provide  efficient,  valid  and  reliable 
information  available  to  facilitate  the  types  of  instructional  processes  that  accompany  active,  engaged  learning 
experiences.  The  research  studies  identified  the  importance  of  designing  flexible  technology  systems  capable  of 
supporting  open-ended  instructional  approaches  and  providing  contextual  feedback  to  a wide  variance  of  learners. 
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Appendix  A 

These  guidelines  emerged  from  analysis  of  the  data  during  the  first  three  weeks  of  the  six-week  study. 
Teachers  were  given  updated  guidelines  on  a weekly  basis.  The  primary  researcher  generated  the  guidelines  from 
analysis  of  the  incoming  key  incident  reports  each  week,  until  the  key  incident  crik  ia  reached  saturation  point. 
Each  key  incident  addressed  the  following  five  questions. 

Question  1:  Describe  the  conditions  in  which  your  monitoring  occurred.  Elaborate  on  these  conditions  of  the 
monitoring  activity  with  respect  to  the  aspect(s)  of  the  assignment  or  the  writing  problem(s)  your  monitoring 
addressed. 

• identify  student's  prior  knowledge  of  concepts  in  the  assignment; 

• explain  student's  prior  use  of  computer  functions  in  the  assignment; 

9 explain  your  familiarity  with  and  purpose  for  using  the  assignment; 

• indicate  when  other  students  listened  to  or  observed  the  key  incident. 

Question  2:  Describe  your  interactions  with  the  student  that  embody  your  monitoring  activity.  Elaborate  on  the 
meaning  of  these  interactions  captured  by  the  dialogue  on  the  tape  recording. 

• identify  who  initiated  the  interaction; 

• explain  why  the  interaction  was  initiated; 

• describe  the  focus  of  the  interaction. 

Question  3:  Describe  the  monitoring  strategies  and  tactics  you  used  to  instruct  the  student.  Elaborate  on  these 
strategies  and  tactics  that  you  intentionally  used  to  intervene  in  the  student's  writing  process. 

• use  action  verbs  that  describe  your  monitoring  activities,  i.c.  roaming,  circulating,  observing,  reacting, 
intervening,  questioning,  probing,  coaching,  directing,  prompting,  supporting,  motivating,  suggesting, 
exploring,  explaining,  simulating,  modeling,  negotiating,  goal-setting,  mediating,  challenging,  etc. 

• describe  any  combination  of  strategics  you  used  with  the  student; 

• define  each  strategy,  i.c.  the  substance  of  the  monitoring; 

• describe  your  purpose  for  using  each  strategy,  i.e.  to  detect,  to  diagnose,  and/or  to  descriptively  or  prescriptively 
treat  a problem  the  student  is  experiencing. 

Question  4:  Describe  the  consequences  of  your  monitoring  that  arc  evident  from  the  student's  drafts.  Elaborate  on 
these  consequences  apparent  in  changes  that  occurred  in  the  student’s  draft  as  a result  of  your  monitoring  activity. 

• focus  on  the  cffect(s)  of  your  monitoring  on  the  student's  computer- generated  writing. 

Question  5:  Describe  any  self-monitoring  you  observed  the  student  engaging  in  as  a consequence  of  your 

monitoring  activity.  Elaborate  on  the  student's  self-reflection  about  the  writing  problem  as  a consequence  of  your 
monitoring. 

• use  action  verbs  such  as  articulating,  reflecting,  exploring,  goal-seeking,  goal-scitrng,  goal-achieving, 
regulating,  planning,  organizing,  generating,  revising,  evaluating,  transferring  knowledge,  etc. 
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Abstract 

The  paper  attempts  to  reexamine  the  concept  of  reflection  and  reflective  thinking  as  a means  of  developing 
expertise  in  instructional  designers.  It  is  argued  that  since  the  real  world  problems  that  designers  face  are  ill- 
structured  and  ill-defined,  designers  must  become  reflective  thinkers  in  order  to  define  the  problem,  evaluate  and 
examine  relevant  information  and  construct  a plausible  solution,  acknowledging  that  the  solution  itself  is  open  to 
further  evaluation  and  scrutiny.  In  an  attempt  to  describe  conditions  and  strategies  for  promoting  ref  active 
thinking,  two  conceptual  models  are  proposed  and  discussed. 

The  need  for  the  reflective  instructional  designer 

Most  of  the  problems  that  instructional  designers  confront  in  their  practice  are  unclear,  unique  and 
situation-based,  and  cannot  be  described  with  a high  degree  of  completeness  or  solved  with  a high  degree  of 
certainty.  These  problems  are  different  from  well-structured  problems  that  can  be  described  completely  and  the 
solution  can  be  identified  as  true  or  correct.  In  contrast  to  well-structured  problems,  ill-structured  problems  require 
the  thinker  to  consider  alternative  arguments,  seek  out  new  evidence,  or  evaluate  the  reliability  of  data  and  sources 
of  information  (King  & Kitchner,  1994).  In  order  for  designers  to  choose  between  competing  ends  and  being  able  to 
solve  the  ill-structurcd,  ambiguous  and  situation-bas 'd  problems,  they  must  reach  to  a level  of  thinking  that  is 
complex  and  reflective. 

Simple  logic  or  procedures  arc  not  adequate  for  solving  the  kind  of  problematic  situations  that  arc  highly 
complex  and  context-based.  Kitchner  and  Kitchner  (1981)  argue  that  in  order  to  solve  an  ill-structurcd  problem,  the 
thinker  must  be  reflective.  A reflective  thinker  should  hold  the  epistemological  assumption  that  each  solution  may 
have  some  validity  and  may  contain  some  errors,  and  that  there  may  be  no  absolutely  correct  choice.  The  basic 
difference  between  this  type  of  thinking  and  authority-based  thinking  or  formal  logic  is  that  the  first  requires  the 
continual  evaluation  of  beliefs,  assumptions  and  hypotheses  against  existing  data  and  against  other  plausible 
interpretations  of  the  data.  It  requires  abductivc  thinking  and  reasoning.  The  resulting  judgments  arc  offered  as 
reasonable  integration  or  syntheses  of  opposing  points  of  view,  not  a correct  solution.  This  type  of  complex 
thinking  is  what  Dewey  also  called  reflective. 

The  importance  of  reflective  thinking  skills  as  a key  element  in  designer’s  expertise  and  professional 
development  is  being  recognized  in  the  field  of  educational  technology  (e.g.,  Rathbun,  Saito  & Goodrum,  1997; 
Roland,  1994;  Willis,  1995).  New  design  practices  and  proposed  design  models  which  arc  based  on  constructivist 
interpretivist  theory  identify  reflection  as  a major  component  of  the  processes  of  the  decision  making  in  design 
activities  (Moallem  & Earle,  in  press;  Willis,  1995).  Carefully  articulated  experiences  that  provide  strategics  for 
linking  knowledge  and  action  through  rcfleciion  arc  becoming  more  common  in  the  educational  technology  field. 
However,  there  has  not  been  enough  discussion  and  evidence  about  how  to  foster  reflective  thinking  in  designers. 
While  it  is  obvious  that  using  a recursive  design  model  requires  a reflective  instructional  designer  who  is  able  to 
think  reflectively  and  reason  adbuclively,  more  efforts  arc  devoted  to  developing  design  models  rather  than  to 
training  reflective  designers. 

The  purpose  of  this  paper  is  to  reexamine  reflective  thinking  from  different  perspectives  and  upon 
rcconccptulization  of  the  term,  propose  a model  for  fostering  reflective  thinking  and  judgment  among  instructional 
designers.  The  paper  also  provides  practical  applications  of  the  proposed  reflective  model  in  practice. 


Reflective  thinking:  A re-examination 

Reflection  as  controlled  thinking 

Dewey,  the  founder  of  the  concept  of  reflective  thinking  in  education,  defines  reflection  as  “active, 
persistent,  and  careful  consideration  of  any  beliefs  or  supposed  form  of  knowledge  in  the  light  of  the  grounds  that 
support  it  and  the  further  conclusions  to  which  it  tends”  (1933,  p.  9).  He  explains  that  the  origin  of  reflective 
thinking  is  a felt  difficulty,  a state  of  doubt  or  perplexity,  which  follows  with  the  suggestion  of  some  way  out-the 
consideration  of  some  solution  for  the  problem.  The  data  at  hand,  he  says,  cannot  supply  the  solution;  they  can  only 
suggest  it.  What  then  are  the  sources  of  the  solution?  Dewey  (1933)  indicates  that  past  experience,  a fund  of 
relevant  information,  whether  or  not  the  thinker  has  some  acquaintance  with  similar  situations  and  materials  at  hand 
may  help  him/her  to  find  some  way  out.  However,  unless  there  has  been  some  analogous  experience,  confusion 
remains  which  urges  the  thinker  to  think  and  weigh  the  grounds  on  which  the  solution  rests.  Dewey  further 
distinguishes  between  ordinary  thinking  and  reflective  thinking  by  emphasizing  that  the  latter  includes  a “conscious 
and  voluntary  effort  to  establish  belief  upon  a firm  basis  of  evidence  and  rationality”  (Dewey,  1933,  p.  16). 

Similarly,  in  an  analysis  of  reflective  thought  process,  Hullfish  and  Smith  (1961)  define  reflection  as 
controlled  thinking  which  is  guided  by  purpose  and  tends  to  be  recursive.  Basing  their  views  on  epistemological 
analysis  that  defines  knowing  as  the  intcrelatcdness  of  memory,  imagination  and  scnliency,  Hullfish  and  Smith 
argue  that  “the  raw  data  of  experience  (pure  sentiency)  arc  not  objects  of  knowing”  (p.  34).  Rather  that  the 
interpretation  of  the  raw  data  which  is  the  result  of  the  overtone  of  memory  (past  experiences)  and  imagination  is. 
In  their  view,  when  the  cues  arc  meaningful  (that  is,  the  more  related  to  the  past  experience)  and  recognition  takes 
place  quickly  (instantaneous  function  of  thinking  which  they  call  “a  synthetic  function”),  there  is  no  need  to  exercise 
conscious  control  of  the  thougnt  process.  But  when  recognition  fails,  even  momentarily,  the  situation  calls  forth 
feelings  of  uncertainty  and  doubt  which  requires  the  individual  to  become  aware  of  both  “the  analytic  and  synthetic 
functions  of  thinking”  (p.  74).  Thus,  as  with  Dewey  (1933),  Hullfish  and  Smith  call  this  feeling  of  uncertainty  and 
doubt  “the  ground  from  which  reflective  thinking  arises  and  a ground  which  intermingles  emotional  involvement 
and  reflective  thinking”  (1961,  p.36). 


Reflection  and  tacit  knowledge 

In  a more  recent  conceptualization  of  reflective  thinking,  Schon  (1987)  expands  on  Dewey’s  definition  of 
reflective  thinking  by  emphasizing  the  tacit  or  intuitive  nature  of  knowledge  and  the  ways  of  revealing  it  through 
self-reflection.  Schon  (1987)  speaks  of  intuitive  knowledge  with  a special  focus  on  the  knowledge  that  is  tacit 
(knowledge  in  action).  As  compared  to  decision  making  models  or  even  Dewey’s  notion  of  reflection,  this  type  of 
knowlcdgc/action  activity  docs  not  rely  on  a scries  of  conscious  steps  in  a decision-making  process.  This  knowledge 
is  inherent,  instead,  in  aciion:  it  is  based,  in  part,  on  the  past  experience  of  the  practitioner  interacting  with  the 
particular  situation.  Schon  suggests  that  knowledge  in  action  is  dynamic  and  unconsciously  controls  how  we  act  in 
relation  to  problem  situations.  Schon  (1987),  also  confirms  that  reflection  occurs  when  there  is  a surprise,  when 
something  interrupts  the  flow  of  knowlcdgc-in-aclion  that  guides  our  everyday  activity. 

However,  Schon  broadens  the  possibilities  of  reflection  by  suggesting  that  we  deal  with  the  surprise  or 
feeling  of  uncertainty  in  two  ways:  “rcflection-in-aclion”  and  “reflcction-on-action”  (Schon,  1987,  P.  35). 
Rcflcction-in-action  occurs  while  the  thinker  is  engaged  in  an  activity  and  is  confronted  with  a surprise.  In  such 
case,  the  person  frames  and  refrumes  the  problem,  comprehends  its  context,  criticaf.y  evaluates  the  underlying 
assumptions  that  led  to  the  problem  and  constructs  an  alternative  method  of  solving  it  which  can  be  tested  on  the 
spot  (Schon,  1987).  Schon  further  distinguishes  two  forms  of  rcflcction-in-action.  In  the  more  formulatic  form,  the 
thinker  uses  the  existing  rules  and  procedures  of  practice  to  solve  the  problematic  situation,  and  in  the  more  elusive 
form,  the  thinker  responds  to  the  surprising  findings  by  inventing  new  rules  on  the  spot.  Reflcction-on-action,  on 
the  other  hand,  is  retrospective  and  takes  place  after  a decisive  action.  The  major  difference  between  rcflcction-in- 
action  and  rcflcction-on-action,  from  Schon’s  perspective,  is  that  the  latter  docs  not  have  an  effect  on  present  action 
and  the  solution  remains  hypothetical  until  it  is  implemented. 

Reflection  and  epistemic  assumption 

Recognizing  that  real  world  problems  arc  complex  and  ill-slructurcd,  King  and  Kitchncr  (1993)  elaborate 
on  Dewey’s  and  Schon’s  notion  of  reflective  thinking  by  suggesting  that  situations  may  be  problematic  for  a variety 
of  reasons,  such  as  the  unavailability  of,  or  the  impossibility  of,  attaining  data  necessary  for  solving  the  problem. 
Individuals  who  do  not  recognize  that  such  situations  arc  truly  problematic  cannot  make  reflective  judgments  (King 
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& Kitchncr,  1993).  Emphasizing  cpistcmic  assumptions  required  Tor  reflective  thinking.  King  and  Kitchncr 
distinguish  between  reflective  judgment  and  critical  thinking,  two  terms  that  arc  sometimes  used  interchangeably, 
even  by  Dewey  (1933,  1938). 

King  and  Kitchner  argue  that  critical  thinking  focuses  on  inductive  and  deductive  logic  skills,  and  assumes 
that  learning  logic  or  general  problem  solving  principles  and  knowing  how  to  use  them  will  lead  to  critical  thinking. 
By  contrast,  King  and  Kitchner  explain,  “rather  than  logic,  basic  differences  in  assumptions  about  what  can  be 
known  and  how  knowing  occurs  differentiate  authority-based  thinkers  from  those  who  use  reflective  thinking” 
(King  and  Kitchner,  1993,  p.  9).  From  their  perspective,  deductive  and  inductive  logic  cannot  totally  account  for  a 
naturally  occurring,  rational  problem  solving  because  a person  may  argue  validity  using  formal  logic,  but  still  use 
authority  as  the  basic  criterion  for  truth.  They  further  argue  that  problems  that  can  be  solved  using  deductive  and 
inductive  logic  can  also  be  described  with  a high  degree  of  completeness,  certainty  and  correctness  (well-defined). 
Such  problems  do  not  require  considering  alternative  arguments,  seeking  out  new  evidence,  or  evaluating  the 
reliability  of  data  and  sources  of  information  (Kitchner,  1983).  However,  real  world  problems  that  require  reflective 
thinking  are  far  from  well-structured  (Churchman,  1971).  The  solution  to  the  ill-struclurcd  problems  depends  on 
how  well  the  thinker  is  able  to  evaluate  the  logical  strength  of  one  frame  of  reference  against  another. 

Reflection  and  abduction 

Along  the  same  line,  Hullfish  and  Smith  point  to  the  power  of  abductive  reasoning  for  solving  ill- 
structured  problems.  They  argue  that  while  most  of  us  are  familiar  with  deductive  and  inductive  reasoning, 
unfortunately,  we  tend  to  view  these  inferences  as  separate  types  of  thinking  instead  of  viewing  them  as  movements 
arising  within  the  total  reflective  process.  They  explain  the  differences  between  abductive  inference  and  two  other 
forms  of  inferences  as  follows. 

. . In  the  situation  calling  for  deduction  wc  are  confronted  with  two  propositions  believed  to  be  related  by 
implication.  Our  reason  for  reasoning  is  to  infer  the  truth  or  falsity  of  one  from  the  truth  or  falsity  of  the 
other.  In  the  situation  calling  for  induction  we  are  confronted  with  materials  which  are  believed  to  be  a fair 
sample  of  a large  class  of  familiar  material.  Our  reason  for  reasoning  in  this  instance  is,  to  infer  what  is 
safely  assertable  about  the  whole  class  from  what  may  be  discovered  about  the  sample.  But  in  the  case  of 
abduction,  we  arc  confronted  with  facts,  events,  situations,  and  the  like  believed  to  be  relevant  to  some 
interest,  question,  or  problem.  Our  reason  for  reasoning  now  is  to  create,  by  inference,  possible 
explanations  for  these  facts  (p.  120). 

Hullfish  and  Smith  conclude  that  hypothesis  formation  during  reflective  thinking  process  or  abductive 
inferences  is  our  initial  hunch  or  idea,  which  constitutes  our  hypothetical  solution  for  the  problem  raised.  Using 
abductive  inference,  we  attempt  to  order  what  is  observed.  In  our  attempt,  we  do  not  assume  the  existence  of 
anything  whose  operation  could  not  be  observed  and  verified.  It  is  thus  that  hypotheses  arc  created,  and  these 
hypotheses  direct  the  investigation  that  leads  to  the  desired  ordering.  Within  the  investigation,  Hullfish  and  Smith 
attest,  both  inductive  and  deductive  inferences  may  function  (Hullfish  & Smith,  1961 ). 

Furthermore,  in  the  case  of  abduction,  there  is  almost  no  guarantee  for  certainly,  and  so  the  principles  of 
sound  abduction  in  fact,  arc  the  principles  of  sound  guessing  (Shank,  1987).  The  guess  or  hypothesis  that  we  select 
as  the  best  is  not  necessarily  the  very  best  possible  hypothesis  on  the  subject,  but  simply  the  best  that  wc  have  based 
on  the  available  data.  In  other  words,  when  confronted  with  an  ill-structurcd  problem,  we  find  that  many  hunches  or 
suggestions  or  hypotheses  presents  themselves,  and  wc  must  select  among  them.  In  choosing  among  the  hunches 
that  present  themselves,  we  progressively  eliminate  or  modify  any  suggestion  that  does  not  account  for  the  facts 
which  would  have  to  be  regarded  as  relevant  if  the  hunch  is  to  be  accepted  as  a working  base.  This  type  of  thinking 
and  reasoning  calls  attention  to  the  fact  that  our  knowledge  of  factual  affairs  is  subject  to  possible  revisions  or 
rejection  in  the  light  of  new  evidence  (all  factual  assertions  arc  subject  to  modification  as  new  evidence  develops). 
It  also  indicates  that  our  theoretical  accounts  arc  never  finally  verified  and  are  subject  to  revisions  or  rejection  when 
someone  creates  another  way  of  structuring  experience  that  proves  to  he  even  more  useful.  This  form  of  thinking 
and  inferencing  may  also  be  contrasted  with  the  form  in  which  one  accepts  a belief  uncritically  by  relying  upon 
authority  without  recognizing  the  modifications  which  new  facts  must  introduce  (Columbia  Associates  in 
Philosophy,  1923;  King  & Kitchncr,  1993). 
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Reflection:  A willingness  to  act 

Reflective  thinking  is  an  emotional  as  well  as  rational  process  (Dewey,  1933;  Hullfish  & Smith,  1961, 
1966;  King  & Kitchner,  1993).  Both  Dewey  and  Hullfish  and  Smith  argue  that  emotions  are  central  to  reflective 
thinking.  The  individual  becomes  involved  in  only  those  problems  he  accepts  to  solve.  Inevitably,  therefore,  “his 
biases,  knowledge  and  values  are  vital  factors  in  the  reflection  he  carries  on  in  problem  situations”  (Hullfish  & 
Smith,  1961,  p.  1 09).  Although  reasoning  gives  the  individual  the  ability  to  rise  above  any  pattern  of  belief  which  is 
formed  as  a result  of  an  unthinking  or  conditioning  process  (prejudice),  the  removal  of  prejudice  is  no  simple  matter. 
Prejudices  arc  rooted  in  such  deep  feelings  that  it  does  not  occur  to  us  to  question  them.  Due  to  our  deep  emotional 
attachment  to  a belief,  we  may  avoid  reflective  construction  of  our  experience  by  engaging  in  the  act  of 
rationalization.  Sometimes  rationalization  takes  the  form  of  gathering  considerable  evidence  to  prove  congruency 
for  parts  of  the  beliefs  that  were  not  incongruent  in  the  first  place.  Therefore,  reflection  requires  the  thinker  to  be 
inclined  to  be  thoughtful  or  reflective  about  the  problem. 

Reflection  as  a social  practice 

In  a most  recent  analysis  of  reflective  thinking,  Zcichner  and  Liston  (1996)  criticize  the  way  Dewey, 
Schon  and  others  conceptualized  reflective  thinking.  They  argue  that  reflective  thinking  has  been  portrayed  as  a 
highly  individualistic  affair  involving  the  thinker  and  the  situation,  and  not  a social  process  taking  place  within  a 
learning  community.  Zeichner  and  Liston  and  several  other  scholars  (Osterman  & Kottkamp,  1993;  Solomon, 
1987),  therefore,  introduce  the  idea  of  reflection  as  a social  practice  and  make  the  argument  that  our 
ideas/hypotheses  become  more  real  when  wc  can  speak  about  them  to  others.  They  further  argue  that  the  thinker 
should  also  be  encouraged  to  reflect  on  the  social*  context  within  which  the  problem  situation  exists.  Recognizing 
the  social  context,  they  suggest,  will  lead  the  thinker  to  see  how  his  problem  situation  is  linked  to  other  problems 
that  exist  in  the  institutional  and  social  context. 

In  sum,  it  seems  that  what  sets  reflective  thinking  apart  from  other  forms  of  problem  solving  and  critical 
thinking  is  an  individual’s  ability  to  not  only  reason  abductively  (Hullfish  & Smith,  1961),  but  to  willingly  examine 
one’s  own  beliefs  or  assumptions  about  the  source  of  knowledge  and  the  meaning  of  “truth”  when  faced  with  ill- 
structured  problems  (King  & Kitchner,  1993).  Furthermore,  reflective  thinking  is  an  individual  and/or  social 
attempt  to  reorganize  or  reconstruct  self  as  well  as  past  experiences.  It  is  the  process  by  which  the  individual 
(sometimes  in  dialogue  with  others)  comes  to  new  understandings  of  his  self,  his  actions,  and  his  taken-for  -granted 
assumptions. 

A Reflective  Thinking  Model 

Figure  1 presents  a spiral  model  that  summarizes  the  process  and  components  of  reflective  thinking 
discussed  in  the  previous  section.  The  mode!  assumes  that  reflective  thinking  is  an  insequential  but  purposeful 
mental  activity.  As  shown  in  the  model,  according  to  Dewey,  Schon  and  others,  there  arc  five  phases  of  reflective 
thoughts  (Columbia  Associates  in  Philosophy,  1923,  Dewey,  1933;  Hullfish  & Smith,  1961,  1966;  Glaser,  1972; 
Schon,  1987):  (1)  Problem  recognition  or  problem  setting,  (2)  problem  clarification  or  what  Dewey  called 
‘‘intellectualization”  of  the  problem,  (3)  hypotheses  or  suggestion  formation  and  modification,  (4)  mental  elaboration 
of  suggestions,  and  (5)  action  taken  on  the  basis  of  the  best  supported  hypothesis  or  suggestion. 

While  these  phases  describe  mechanics  of  the  reflective  process,  six  components  explain  the  nature  and 
dynamics  of  reflective  thinking  process:  (1)  Willingness  to  be  thoughtful,  (2)  willingness  to  rccognizx  and 
understand  the  context  within  which  assumptions  and  the  actions  are  formed,  (3)  willingness  to  explore  and  imagine 
alternatives,  (4)  the  understanding  and  accepting  of  real  uncertainty  (cpistcmtic  assumptions),  (5)  using  abductivc 
inference,  and  (6)  being  exposed  to  a variety  of  interpretive  considerations  (in  dialogue  with  others).  The  phases 
and  components  of  reflective  thinking  arc  interdependent.  In  other  words,  the  ability  to  engage  in  the  reflective 
thinking  process  depends  on  whether  or  not  the  components  of  reflective  thinking  exist. 

As  Figure  1 shows,  reflection  happens  only  if  the  designer  is  inclined  to  be  about  his/her  beliefs.  While 
thinking  is  natural,  reflective  thinking  is  not.  A designer  must  he  willing  to  consider  his  beliefs  and  engage  in 
reflective  thinking  (Dewey,  1933;  Grimmett,  MacKijnon,  Erickson,  & Ricckcn,  1990;  Hullfish  & Smith,  1961 ; King 
& Kitchner,  1993).  Without  the  disposition  to  engage  in  reflective  thinking,  designers  will  not  be  able  to  see  issues 
from  their  client’s  view  point.  Furthermore,  true  reflective  thinking  presupposes  that  the  designer  holds  the 
cpistcmic  assumptions  that  allow  him/her  to  understand  and  accept  the  real  uncertainty  and  differentiate  between 
well-  and  ill-structured  problems  of  practice  (King  & Kitchner,  1993). 


Willingness  to  recognize  and  understand  the  context  within  which  designer’s  assumptions  and  the  actions 
arc  formed  is  a major  component  of  reflection  (Brookfield,  1987;  Zcichncr  & Liston,  1996).  Contextual  designers 
view  their  dearly  held  beliefs  and  values  as,  to  some  extent,  social.  They  understand  that  value  systems  and 
behavioral  codes  arc  socially  transmitted  as  well  as  personally  generated  by  personal  experiences.  They  arc  willing 
to  acknowledge  that  their  decisions  and  actions  have  an  effect  on  their  social  context.  The  designer’s  willingness  to 
realize  that  his/her  personally  devised  values,  beliefs,  and  moral  codes  are,  in  fact,  culturally  induced,  results  in 
forming  hypotheses  or  suggestions  that  arc  context  specific  and  broader  in  perspective. 

Central  to  reflective  thinking  is  the  designer’s  capacity  to  explore  and  imagine  alternatives  to  solve  a 
problem  situation  (forming  hypotheses  and  mental  elaboration  of  hypotheses).  When  the  designer  realizes  that  there 
arc  alternative  hunches  (guesses,  hypotheses)  to  a problem  situation,  he/she  is  skeptical  of  the  claim  to  universal 
truth  or  to  ultimate  explanation.  Such  a designer  scrutinizes  and  clarifies  the  problem  by  attending  to  the  facts, 
events  and  situations  that  he/she  believes  to  be  relevant  to  the  problem  (abductivc  inference)  instead  of  jumping  to 
an  inductive  generalization.  Of  course,  the  question  of  which  facts  are  truly  relevant,  is  a matter  of  inquiry.  When 
confronted  with  a performance  problem,  a reflective  designer  has  many  hunches  or  hypotheses.  He/She  then 
chooses  among  the  hunches  that  present  themselves  by  eliminating  or  modifying  any  suggestion  that  docs  not 
account  for  the  facts  which  would  have  to  be  regarded  as  relevant  if  the  hunch  is  to  be  accepted  as  a working  base 
(Hullfish  & Smith,  1961).  However,  the  reflective  designer  realizes  that  the  chosen  hypothesis  is  still  hypothetical 
and  subject  to  continual  modification  in  the  process,  or  even  to  replacement.  This  form  of  thinking  which  Shank 
( 1987)  calls  “acts  of  skill”  instead  of  reasoning,  (p.  279),  can  be  the  source  of  the  designer’s  creativity  (Peirce,' 1955) 
because  it  helps  him/her  to  move  back  and  forth  from  facts  to  theory  and  theory  to  facts. 

On  the  basis  of  the  best  supported  hypothesis  or  hunch,  a reflective  designer  must  take  an  action  and  seek 
confirming  evidence,  which  can  leave  him/her  to  make  the  judgment  or  come  to  a resolution  for  the  problem.  As 
Schon  defines  in  his  reflection-in  and  on-action,  the  designer  should  have  the  opportunity  to  experiment  with  the 
actual  operation  of  his/her  solution  before  determining  practice  (reflection-in-action).  However,  there  arc  times  that 
the  designer  cannot  experiment  with  the  actual  operation  of  his/her  conclusion  (reflcction-on-action).  In  such  cases, 
the  closing  of  reflective  process  incident  comes  only  after  the  experimental  verification  (Schon,  1987). 

Three  aspects  of  reflection:  From  theory  to  practice 

Emphasizing  the  importance  of  reflection  as  a key  clement  of  designers'  creativity  and  professional  growth,  the 
preceding  model  was  used  to  identify  three  aspects  that  arc  essential  in  fostering  reflective  thinking  in  instructional 
designers.  These  three  aspects  arc; 

* Self-reconstruction  (understanding  of  assumptions  underlying  thought  and  action) 

* Action-reconstruction  (new  understanding  of  the  problem  situation) 

* Social-reconstruction  (reexamining  taken- for-granted  assumptions  about  design  and  development) 

For  each  of  these  aspects,  it  is  assumed  that  the  source  of  knowledge  for  reflection  is  both  found  in  the  context  of 
action  and  in  the  practical  application  of  personal  knowledge.  It  is  also  assumed  that  puzzlement  and  subsequent 
reflection  about  a situation  or  presuppositions  that  guide  action  in  it,  lead  to  a mode  of  knowing  described  by 
Grimmett  and  others  (1990)  as  ‘‘dialectical”  (p,  27).  In  this  view,  knowledge  is  seen  as  emergent,  that  is  the 
designer  reframes  and  reconstructs  past  undcrstai  ding  in  such  a way  as  to  generate  a fresh  appreciation  of  a practice 
situation  (Schon,  1987).  Each  of  the  three  aspects  are  subsumed  under  these  assumptions. 

Self-reconstruction 

This  aspect  of  reflection  is  focused  on  the  individual's  view  of  himsclf/hersclf  as  a designer  (what  it  means 
to  be  an  instructional  designer).  In  this  aspect,  future  designers  perform  a personal  search  to  become  more  aware  of 
the  cultural  milieu  in  which  they  operate.  Van  Mancn  (1987)  describes  three  ways  in  which  the  idea  of  self- 
reflection  can  be  used:  self-reflection  as  an  ontological  phenomenon,  sclf-rcflcction  as  a life  philosophy,  and  self- 
reflection  as  a methodological  concept.  As  an  ontological  phenomenon,  self-reflection  is  concerned  with  ways  of 
being  in  the  world  (Grimmett  ct.al,  1990)  and  with  how  wc  come  to  understand  our  own  existence.  Self-reflection 
as  a life  philosophy  focuses  on  individuals’  understanding  of  themselves  through  life  and  life  experiences.  Finally, 
self-rcflcdion  as  a methodological  concept  refers  to  being  attentive  and  thoughtful  to  the  relationship  between 
theory  and  practice.  Self-reflection  from  this  perspective  means  being  able  to  develop  and  provide  warrants  for  our 
own  beliefs  and  interpretations.  Generally  speaking,  the  three  levels  of  sclf-reflcction  provide  designers  with  an 


opportunity  to  shape  and  restructure  their  tacit  knowledge  and  beliefs  about  design  and  development  which  will 
directly  impact  the  way  they  think  and  the  way  they  make  judgments  and  decisions. 

Action-reconstruction 

In  action  reconstruction,  designers  reframe  actual  problem  situations  in  a manner  which  leads  to  new 
insights  about  design  theories.  The  process  of  reframing  the  experience  will  lead  to  a new  way  of  seeing  the 
situation  (learning  to  think  abductively).  Based  on  this  aspect  of  reflection,  designers  learn  to  examine  the  existing 
problems  of  design  practice  abductively  and  conceptualize  needed  changes  based  on  reflcction-in  and  on-action.  In 
this  aspect  of  reflection,  designers  learn  to  understand  and  examine  a variety  of  contexts  ranging  from  face-to-face 
interaction  with  expert  designers,  developers  and  clients  to  social  and  political  forces  influencing  what  goes  on  in 
settings.  In  order  to  understand  and  deal  with  situations,  designers  identify  the  cultural  context  cf  the  problem  and 
engage  in  generating  alternative  perspectives. 

Social-reconstruction 

Social-reconstruction  refers  to  the  aspect  of  reflection  that  requires  designers  to  identify  and  address  the 
social,  political,  and  cultural  conditions  that  frustrate  and  constrain  empowerment  and  self-control.  This  aspect  of 
reflection  discovers  those  beliefs  or  behaviors  which  preserve  the  inadequacies  of  the  current  system  and  prevents 
the  introduction  of  new  ideas  and  better  approaches  in  instruction.  By  involving  designers  in  critical  reflection, 
designers  will  make  explicit  for  themselves  the  intentions  and  underlying  assumptions,  knowledge,  values,  and 
sensitivities  that  guide  the  design  practice. 

Promoting  reflective  thinking  in  instructional  designers:  Strategies  and  suggestions 

Bach  of  the  three  aspects  of  the  reflective  thinking  approach  suggests  conditions  and  strategies  that  can 
promote  different  components  of  the  reflective  thinking  process.  The  reflective  thinking  approach  differs  from  the 
traditional  learning  theory  approach  in  developing  expertise  in  designers.  While  in  traditional  learning  theory 
approach,  specific  techniques  of  analysis  and  learned  rules  of  practice  arc  emphasized,  the  proposed  reflective 
thinking  approach  is  grounded  in  the  designers’  own  internal  rules  for  making  sense  of  the  experiences. 
Furthermore,  the  reflective  thinking  approach  to  developing  expertise  includes  designers’  personal  understandings  of 
practice  situations.  This  form  of  knowledge  construction  requires  an  engagement  in  authentic  cognitive  activities  in 
which  the  novice  designer  is  challenged  by  a mentor  through  constant  dialogue  and  sharing  of  the  experiences.  This 
kind  of  environment  not  only  resembles  the  every  day  social  setting  in  workplaces,  but  it  also  helps  designers 
become  reflective  in  the  analysis  of  their  own  actions  through  the  cycle  of  reflection  consisting  of  problem  setting, 
reframing  (clarifying  the  problem)  and  resolution.  This  form  of  knowledge  construction  also  requires  a learning 
environment  in  which  novice  designers  arc  constantly  encouraged  to  question  the  validity  of  their  presuppositions 
and  the  relationships  between  these  assumptions  and  the  social,  political  and  cultural  conditions.  Thus,  a cognitive 
apprenticeship  model  (Collin,  Brown,  Newman,  1989)  accompanied  by  assistance  or  “scaffolding”  from  mentors, 
creates  the  best  conditions  for  reflective  thinking.  The  following  section  looks  at  a number  of  strategies  that  can  be 
used  to  promote  reflective  thinking  in  instructional  designers.  It  must  be  noted  that  the  role  of  mentor  or  expert 
designer  is  critical  in  promoting  all  aspects  of  reflective  thinking. 

Strategies  to  facilitate  self-reconstructson 

Self-reconstruction  can  be  best  facilitated  within  an  apprenticeship  model  in  which  the  mentor  helps  the 
novice  designer  to  reflect  on  his/her  underlying  assumptions  upon  building  trust  and  self-respect.  The  mentor  may 
assign  mysterious  tasks,  introduce  contradictory  ideas,  question  tacit  assumptions,  or  refuse  to  answer  questions  in 
order  to  create  disequilibrium.  The  mentor  can  also  create  the  conditions  in  which  designers  ask  critical  questions 
about  why  people  think  and  behave  in  the  ways  they  do,  how  people  come  to  know  something  (epistemology  of 
knowledge)  and  what  the  relationship  is  between  what  they  think  and  what  they  do.  Another  condition  that  can 
promote  self-reconstruction  is  called  critical  incident,  a technique  that  was  first  introduced  by  Flanagan  more  than 
forty  years  ago  (1954).  The  focus  of  the  critical  incident  is  on  the  events  that  arc  of  particular  importance  to  the 
thinker.  Critical  incidents  provide  expert  mentors  with  hunches  as  to  what  are  the  most  significant  concerns  and 
assumptions  of  the  novice  designers.  They  arc  also  important  because  they  encourage  designers  to  talk  about 
themselves  without  being  consciously  aware  of  it.  For  example  when  a designer  criticizes  other  designers’  actions 
as  inappropriate,  he/shc  is  saying  something  abo  it  his/her  own  conception  of  practice.  Identifying  and  analyzing  the 
validity  of  assumptions  influencing  design  practice  and  critical  debates  arc  also  conditions  that  can  promote  self- 
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reconstruction.  While  critical  analysis  helps  designers  develop  criteria  for  evaluating  their  beliefs  and  assumption, 
critical  debates  will  assist  designers  to  take  an  unfamiliar  perspective  on  an  issue  or  to  explain  in  a sympathetic 
manner  a position  with  which  they  disagree. 

Strategies  to  develop  action -reconstruction 

Although  the  spiral  model  of  reflective  thinking  may  suggest  an  overemphasis  on  the  procedures  of  logical 
thinking,  the  whole  reflective  process  is  far  messier  than  four  or  five  stages.  For  example,  the  designer  may  move 
back  and  forth  between  problem  situation  and  analysis  of  alternatives  many  times  before  reaching  a tentative 
conclusion  or  solution.  Moreover,  anywhere  during  the  process  the  designer  may  have  an  insight  that  suggests  a 
reconfiguration  of  all  parts  of  his/her  previous  deliberation;  after  much  consideration,  no  alternative  may  appear 
warranted,  yet  the  designer  needs  to  choose  an  action;  or  new  circumstances  may  develop  causing  the  designer  to 
temporarily  stop  the  process.  Thus,  reflective  thinking  requires  both  rational  and  intuitive  thought  process. 
Promoting  such  thinking  requires  authentic  activities  in  which  designers  face  the  real  world  problems  of  practice. 
Simulated  cases  in  which  novice  designers  work  in  groups  to  simulate  real  world  problems  and  the  process  of 
solving  them  will  facilitate  their  abductive  reasoning  skills.  Reflective  discussion  must  follow  the  simulation  cases 
to  provide  designers  with  formative  feedback  and  to  encourage  them  to  engage  in  self-assessment.  Varying  both  the 
degree  of  complexity  and  the  uncertainty  of  the  simulated  cases  will  allow  for  a gradual  development  of  reflective 
thinking.  Another  strategy  that  can  provide  the  proper  conditions  to  promote  reflective  thinking  is  participate  in  a 
real  design  activity.  Engaging  in  real  tasks  of  analysis,  design  and  development  and  working  directly  with  clients 
provides  the  best  opportunity  for  novice  designers  to  test  their  theories  in  action.  Such  activities  will  not  only  help 
designers  to  tests  theories,  but  also  to  realize  the  dissonance  between  what  they  say  they  believe  in  and  what  they 
suspect  to  be  true.  During  these  activities,  designers  should  be  encouraged  to  use  ethnographic  or  interpretive 
techniques  and  to  reflect  on  the  reasons  why  specific  theories  are  not  working  and  to  seek  alternative  forms  of 
practice.  Tins  reflcction-in-action  must  be  followed  by  reflection-on-action  in  which  designers  engage  in  identifying 
assumptions  underlying  specific  theories  and  considering  the  fit  between  these  assumptions  and  the  reality  of 
professional  practices.  One  process  that  must  be  emphasized  in  real  world  problem  solving  activities  is  that  of  the 
problem  setting  (Schon,  1983).  Designers  must  realize  that  in  real  world  practice,  problems  are  defined  by  their 
participants  rather  than  external  authorities.  The  problem  setting,  therefore,  is  grounded  in  designers’  contextual 
awareness  of  the  important  aspects  of  the  workplace  life. 

Strategies  to  develop  social-reconstruction 

The  uncovering  of  assumptions  underlying  organizational  cultures,  and  the  discovery  of  the  connections 
between  policy  decisions,  legislator  change,  and  designer’s  decisions  arc  probably  the  most  untouched  areas  of 
reflection.  Designers  must  have  the  opportunity  to  discover  the  issues  of  control  and  power  and  how  they  arc 
affecting  people’s  personal  relationships.  Questioning  the  basis  on  which  individuals,  groups  and  systems  exercise 
power  and  making  connections  between  isolated  incidents  and  broader  political  happenings  is  the  heart  of  the 
reflective  thinking  approach.  Without  this  realization,  design  efforts  will  become  re-active  instead  of  pro-aclivc. 
Activities  such  as  asking  novice  designers  to  speculate  imaginatively  on  alternative  ways  of  distributing  power,  or 
how  hierarchies  might  be  dismantled  and  structures  changed  in  an  organization  could  facilitate  understanding  of  the 
social  and  political  connections  with  individual  design  efforts.  Conducting  an  analysis  of  the  culture  of 
organizations  which  is  based  on  actual  data,  analysis  of  personal  biographies  in  light  of  how  individuals  have 
assimilated  dominant  cultural  values,  providing  experiences  of  practicing  the  democratic  process  of  decision  making 
and  asking  questions  about  why  power  and  wealth  arc  distributed  the  way  they  are,  are  some  of  the  useful  strategies. 

Conclusion 

In  this  paper,  1 have  attempted  to  reexamine  the  concept  of  reflection  and  reflective  thinking  as  a means  of 
developing  expertise  in  instructional  designers.  I have  argued  that  since  the  real  world  problems  that  designers  face 
are  ill-structured  and  ill-defined,  designers  must  become  reflective  thinkers  in  order  to  define  the  problem,  evaluate 
and  examine  relevant  information  and  construct  a plausible  solution,  acknowledging  that  the  solution  itself  is  open 
to  further  evaluation  and  scrutiny.  In  an  attempt  to  describe  conditions  and  strategies  for  promoting  such  think r 
processes,  I proposed  a conceptual  model  which  led  me  to  identify  three  aspects  of  the  reflective  ilm  king  approach 
and  suggest  a number  of  strategies  for  promoting  each  aspect.  The  reflective  thinking  approach  proposed  opens  a 
new  way  of  educating  and  training  instructional  designers  which  is  consistent  with  new  conceptualization  of  the 
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instructional  design  models  and  with  the  constructivist  learning  theory.  Although  much  more  must  be  learned  and 
discussed  in  this  area,  reflective  thinking  approach  has  obvious  implications  for  the  instructional  design  field. 
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Figure  1:  A Model  of  Reflective  Thinking 
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Introduction 

As  a psychological  construct,  locus  of  control  describes  a condition  when  people  enter  into  a situation  with 
certain  expectancies  regarding  the  probable  outcome  of  their  behavior.  Expectancy  and  reinforcement  are  therefore 
brought  together  in  a working  construct.  A determinate  within  this  framework  is  the  degree  to  which  a person  feels 
that  they  possess  (internal  control)  or  lack  (external  control)  power  over  what  happens  to  them  (Rotter,  1966).  The 
use  of  visuals  in  instructional  materials  offers  potential  for  subjects  to  control  their  immediate  learning  environment 
by  selecting  or  attending  to  certain  visual  aspects  within  a visual. 

Research  has  shown  that  color-coding  instructional  materials  helps  learners  organize  or  categorize 
information  into  useful  patterns  which  learners  interpret  and  adjust  to  their  environment.  Color-coding  may  be 
considered  a strategy  with  which  students  enhance  or  sharpen  essential  message  characteristics  by  providing 
structures  for  the  interpretation  of  new  information  (Dwyer,  1978)  If  visualization  and  the  use  of  color-coding 
provides  reinforcement  to  learners  (particularly  those  of  high  or  external  locus  of  control),  color  coded  materials 
may  be  a positive  factor  in  attempts  of  learners  to  control  their  visual  learning  environment.  The  purpose  of  this 
study  will  be  to  examine  the  effects  that  color  (b/w  and  color)  has  on  the  information  processing  strategics  of  the 
internal  and  external  locus  of  controlled  learners.  It  is  the  primary  hypothesis  of  this  study  that  the  color  coding  will 
significantly  improve  the  students  identified  as  being  external  locus  of  control.  It  was  anticipated  that  the  color- 
coding of  concepts  and  processes  would  help  "externals"  recognize  and  encode  imprint  concepts  and  processes 
leading  to  greater  understanding  of  the  information  presented  and  subsequent  achievements. 

Literature  Review 

Locus  of  control  has  been  defined  as  a form  of  self-appraisal  related  to  the  degree  to  which  individuals 
view  themselves  as  having  some  causal  role  in  determining  specific  events.  Rotter's  (1966)  Internal-External  Locus 
of  Control  Scale  separates  individuals  into  two  perceptual  groups  (a)  internals  being  individuals  who  perceive  their 
behavior  as  being  directly  related  to  outcomes,  and  internals  who  perceive  outcomes  as  being  dictated  by  chance, 
luck  or  circumstances  outside  of  their  control  (Rotter,  1966;  Joe,  1971).  Hersh  and  Scheibc  (1967)  and  Gozali  ct.  al. 
(1973)  found  the  reliability  of  the  Internal-External  Locus  of  Control  Scale  to  range  from  0.70  to  0.80. 

Observations  from  prior  research  indicates  that:  externals  preferred  chance  situations  over  skill  situations 
(Feather,  1967);  internals  actively  seek  information  relevant  to  problem  solving  (Davies  & Pharcs,1967).  Internals 
tend  to  retain,  learn,  remember  and  attend  to  more  information,  especially  when  the  information  is  related  to 
personal  future  goals  (Long,  1979).  Internals  are  more  active  information  processors  and  are  able  to  better  use 
previous  information  in  decision-making  tasks  (Pines,  1973).  Internals  also  believe  that  reinforcements  are 
contingent  upon  their  own  behavior,  capacities,  or  attributes.  Externals  believe  that  reinforcements  arc  not  under 
their  personal  control  but  rather  are  under  the  control  of  powerful  others,  luck,  chance^and  fate  (Rotter,  1966; 
Joe,  1971).  Internals  are  generally  more  concerned  with  their  abilities  and  their  failure  (Efran,  1964).  They  seem  to 
have  greater  need  for  independence  and  arc  resistive  to  subtle  attempts  at  influence  (Crowe  & Liverant,  1963; 
Strickland,  1963). 

Statement  of  Problem 

Prior  research  has  found  that  color-coding  to  he  an  important  instructional  variable  in  facilitating  student 
achievement  (Dwyer;  1978,  1987);  its  instructional  effect  on  two  learning  stages  (Internals  and  Externals) 
constituted  the  focus  of  this  study.  Since  students  processing  the  different  learning  styles  perceive  and  process 
information  differently,  different  levels  of  visualization  (black  and  white  and  color  coded)  were  used  in  order  to 
determine  the  most  effective  level  in  facilitating  achievement  for  both  learning  styles.  Specifically,  the  purpose  of 
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this  study  was  to  (a)  examine  the  instructional  effectiveness  of  two  types  of  visualized  instruction  in  facilitating 
student  achievement  for  students  identified  as  possessing  different  styles  of  learning  (Internals  and  Externals);  (b) 
determine  whether  an  interaction  exists  between  types  of  learning  style  and  instructional  treatment,  and  (c) 
determine  whether  specific  instructional  treatments  were  equally  effective  in  facilitating  the  achievement  of  I-E 
students  on  specific  educational  objectives. 

Methods  and  Procedures 

One  hundred  and  seven  college  students  were  classified  as  external,  neutral  or  internal  locus  of  control 
based  on  their  scores  on  the  Rotter  (1966)  Internal-External  Locus  of  Control  Scale.  Students  were  identified  as 
external,  neutral  or  internal  by  separating  the  performance  arc  one  half  standard  deviation  above  and  below  the 
mean.  This  process  resulted  in  thirty-six  students  being  identified  as  externals,  fifty-five  as  neutral  and  twenty  -six 
as  internals.  The  instructional  content  employed  in  the  study  was  the  anatomy  and  functions  of  the  human  heart 
during  the  diastolic  and  systolic  phases.  The  instructional  unit  also  contained  19  visuals  positioned  as  a result  of 
item  analysis.  The  instructional  booklet  included  one  page  of  directions  and  twenty  pages  of  concepts  and  functions 
of  the  heart  integrated  by  prose  text  with  accompanying  visualization.  Directions  on  the  first  page  of  the  booklet 
indicated  to  the  students  that  the  materials  were  part  of  an  investigation  to  study  the  relative  effectiveness  of  visual 
illustrations  accompanying  printed  instructions  and  that  there  was  no  time  limit.  For  purposes  of  this  investigation, 
six  color  categories  and  black  were  used  to  code  the  heart  concepts  (See  Table  1).  Printer's  ink  colors  selected  were 
from  Pantone  Matching  Systems. 

The  six  colors  were:  red  (Pantone— warm  red,  blue  (Pantone  — process  blue),  green  (Pantonc--grccn), 
purple  (Pantone-No.  227),  brown  (Pantone-No.  471)  and  gold  (Pantonc-No.  124).  The  black  (Pantonc-black) 
was  used  for  all  the  non-color  coded  design  elements  within  the  color  coded  materials  and  for  all  the  design 
elements  within  the  black/whitc  material. 

This  scheme  of  color  coding  application  to  central  concept  sequences  and  the  lack  of  the  color  code  in 
peripheral  sequences  insured  that  the  color  code:  (1)  was  used  only  to  emphasize  the  central  concept  being 
presented;  (2)  was  used  to  structure  a large  number  of  heart  concepts  into  smaller  category  groups:  (3)  was  used  to 
differentiate  dissimilar  concepts;  and  (4)  was  used  to  contextually  (physical  form)  and  semantically  (associative 
value)  relate  similar  concepts  or  functions.  The  scheme  of  color  coding  application  to  the  instructional  sequence 
also  insured  that:  (1)  peripheral  learning  was  not  emphasized;  (2)  a contexual  color  cue  was  not  continuously 
conditioned;  and  (3)  an  association  was  not  based  purely  on  semantics. 

Thus  the  black/white  instruction:  (I)  could  not  visually  emphasize  central  from  peripheral  concepts  within 
the  prose  text;  (2)  could  not  visually  structure  a large  number  of  heart  concepts  into  smaller  category  groups;  (3) 
could  not  visually  differentiate  dissimilar  concepts;  and  (4)  could  not  contextually  (physical  form)  relate  similar 
concepts  or  functions.  The  black/whitc  instruction,  while  having  similar  contextual  cues  (word  labels,  arrows,  and 
shaded  areas),  relied  primarily  on  the  learner's  associative  ability  to  categorize  and  relate  word  meanings. 

Treatment  1.  Students  in  this  treatment  received  the  1800  word  instructional  unit  complemented  by  the  19  black 
and  white  visuals. 

Treatment  2.  Students  in  this  treatment  received  the  identical  prose  instructions  and  visuals  as  did  students  in 
Treatment  1;  however,  their  (19)  visuals  were  coded  according  to  the  scheme  presented  in  Table  1 . 
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Table  1.  Scheme  of  Color  Coding  Application  to  Instruction  and  Test  Materials. 


Color  Code  Heart  Concepts u 

Red  right  auricle;  left  auricle;  cpicardium;  auricles;  diastolic;  lowest  pressure;  contract;  relaxation;  arrows*1 

Blue  right  ventricle;  left  ventricle;  myocardium;  ventricles;  systolic;  greatest  pressure;  pressure  is  greatest;  contract; 
contraction;  arrows0;  arrow 

Green  tricuspid  value;  mitral  value;  endocardium;  apex;  valves;  opened;  opens;  partially  open;  partially  closed; 
closed;  closure;  hashing  and  brackets0 

Purple  pulomary  value;  aortic  value;  semi-lunar  valves;  valves;  tendons;  open;  close;  closed 

Blue  superior  vena  cacva;  inferior  vena  cava;  pulomary  veins;  pericardium;  vena  cavas 

Gold  pulomary  artery;  aorta,  aortic  artery;  septum,  arteries 

Black  body;  lungs;  hearth  arrows*;  miseJ1 

Red  ectoderm;  vcnic  value;  arteries 

Blue 

Green  Vascular  space;  parasympathetic 
Purple  semi-lunar  chamber;  semi-lunar  value 

Brown  pulomary  vein;  inferior  vena  cavas;  superior  vena  cavas;  sympathetic;  veins,  extoxin 
Gold  aortic  base;  aortas;  pulomaryarlcries;  cctocardium;  sympatrie;  cardiac  artery;  septic  value 
Black  none  of  these;  misc  J1 


a Most  colored  heart  concepts  had  an  identical  color  arrow(s)  extending  from  the  concept  word  to  the 

concept  visualized;  some  colored  heart  concepts  had  identical  colored  areas  representing  visually  the  concept  word. 
^ Arrows  were  designed  to  visually  illustrate  auricle  contraction. 
c Arrows  were  designed  to  visually  ilustratc  ventricle  contraction. 

^ Arrows  were  designed  to  visually  illustrate  the  first  and  second  contraction  during  the  systolic  phase. 
c Hashing  and  brackets  were  designed  to  visually  illustrate  the  area  representing  the  concept  apex, 
f The  heart  was  visually  represented  by  a simple  line  drawing. 

£ Arrows  were  designed  to  visually  illustrate  blood  How  and  blood  pressure  against  the  heart  values. 

Some  correct  and  distracter  items  were  black  in  the  comprehension  task. 
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Criterion  Measures 

Rotlcr’s  Internal-External  Locus  of  Control  Scale  was  used  to  identify  the  degree  of  externality  among 
students  participating  in  this  study,  The  scale  consists  of  twenty-nine  fixed-choice  items,  six  of  which  arc  filter 
items.  Immediately  upon  completion  of  these  respective  instructional  modules,  each  student  completed  four 
individual  criterion  measures.  Following  is  a brief  description  of  each. 

Drawing  Test 

The  drawing  test  assesses  the  students'  ability  to  construct  the  heart  in  its  proper  context.  The  students  will 
draw  a representalion  of  the  heart  and  label  the  parts  with  numbers  that  correspond  to  a list  of  20  parts  given  the  test, 
for  example,  the  epicardium,  aortic  valve,  septum,  and  pulmonary  veins.  The  test  was  evaluated  for  correct  visual 
placement  of  the  parts. 

Identification  Test 

This  test  evaluates  the  students’  ability  to  identify  the  parts  of  the  heart  from  supplied  drawings  of  the  heart 
with  four  or  five  letter  labels  pointing  to  various  areas  of  the  heart.  The  purpose  of  the  test  was  to  measure  the 
students'  use  of  visual  cue  to  discriminate  different  structures  of  the  heart  and  connect  proper  names  of  the  heart 
with  the  location  of  the  part. 

Terminology 

The  test  was  designed  to  measure  the  siudents'  knowledge  of  facts,  terms  and  definitions.  The  objectives 
measured  can  be  generalized  to  any  content  area  where  basic  elements  arc  a prerequisite  to  the  learning  of  concepts, 
rules,  and  principles. 

Comprehension  Test 

This  test  measures  the  students'  ability  to  evaluate  the  functions  and  the  position  of  certain  parts  at  a 
particular  time  when  the  heart  is  functioning.  The  student  must  understand  the  parts  of  the  heart,  their  function,  and 
the  simultaneous  process  that  takes  place  during  the  systolic  and  diastolic  phases.  The  purpose  of  the  test  is  to 
measure  understanding  in  order  for  the  student  to  explain  some  other  event. 

Total  Criterion  Test 

The  items  (80)  given  in  the  drawing,  identification,  terminology,  and  comprehension  tests  were  combined 
to  give  a total  achievement  score. 

Results  and  Discussion 

The  2x3  analysis  of  variance  design  was  analyzxd  for  its  main  effects  and  interactions.  The  main  effects 
were  the  locus  of  control  and  color  coding.  Tabic  2 illustrates  the  number  of  students  in  each  comparison  along  with 
there  respective  mean  achievement  scores  and  standard  deviations  on  the  total  criterion  test  and  Internal-External 
Locus  of  Control  Scale  for  the  different  comparisons. 

Results  indicated  that  a significant  main  effect  existed  in  favor  of  the  color-coded  treatment  on  the  total 
(F=  11.01,  dp=*/ll  1 lp.<.05)  identification  (F=4.66,df_*/1 1 1,  p.  (.05)  and  the  drawing  (F=45.3,  */11L  p.  <.05) 
criterion  measures.  Insignificant  differences  between  the  black  and  while  and  color  coded  treatments  were  found  to 
exist  in  the  terminology  and  comphrehesnion  tests.  Insignificant  differences  were  found  to  exist  among  levels  of 

Locus  of  Control  (high,  medium,  low)  (F=1.41,  df=2/ll  1,  p.>.05  and  color  coding  and  Locus  of  Control  (F=58, 

df=2/U,  p.>  .05.  The  Locus  of  Control  variable  did  not  have  a significant  main  effect  in  facilitating  student 
achievement  in  terms  of  their  ability  to  generalize  and  perform  on  the  total  criterion  test.  In  general,  the  Locus  of 
Control  variable  dichotomizes  individuals  who  take  responsibility  for  their  own  meaning  and  those  who  do  not. 
These  insignificant  achievement  results  may  be  partially  examined  by  the  fact  that  college  students  were  used  as  the 
experimental  population.  Although  they  were  divided  into  internals  and  externals  based  on  their  performance  on  the 
Rotter  I-E  Scale,  the  dichotomy  itself  may  have  been  artificial.  Since  most  college  students  have  already  exhibited 
internal  characteristics  (being  of  the  belief  that  industrious  behavior-study  will  lead  to  success)  the  division  (one  half 
standard  deviation  above  and  below  the  grand  mean)  many  separated  internals  into  two  groups  called  internals  and 
externals.  Under  these  circumstances  it  was  unrealistic  to  expect  significant  differences  to  reveal  themselves  in 
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terms  of  increased  p -formance.  Another  possible  expectation  for  the  inability  of  the  different  levels  of  locus  of 
control  to  perform  differentially  on  the  total  criterion  test  might  be  the  first  that  students  did  not  realize  that  the 
colored  illustrations  they  were  receiving  were  colored-coded  according  to  a specific  rationale  (system).  Had  they 
been  aware  of  the  system  their  performance  may  have  improved  significantly. 

Table  2.  Means  and  Standard  Deviations  for  all  Comparisons 


N 

M 

SD 

Treatment  1 

58 

59.85 

12.27 

Treatment  2 

59 

50.42 

14.56 

Locus  of  Control 

Low 

36 

54.75 

13.01 

Medium 

55 

53.55 

15.84 

High 

26 

58.85 

11.84 

Locus  of  Control 

(B  & W and  Color  Coding) 

Color/Low 

20 

58.4 

11.81 

Color/Medium 

26 

60.00 

13.10 

Color/High 

12 

61.92 

11.88 

B & W Low 

16 

50.18 

13.36 

B & W Medium 

29 

47.75 

16.04 

B & W High 

14 

56.21 

11.57 

The  significant  main  effect  indicating  that  students  who  received  the  color  coded  instructional  treatment 
achieved  significantly  higher  scores  on  the  identification,  drawing  and  total  criterion  test  than  did  those  students  who 
received  the  black  and  white  indicates  that  color  remains  to  be  a important  instructional  variable  in  improving 
student  achievement.  The  findings  of  this  study  emphasize  the  importance  of  the  interrelatedness  of  variables 
associated  with  learning  and  the  effective  use  of  visualization  in  the  teaching  learning  process.  It  verifies  the  fact 
that  visualization  in  the  instructional  process  functions  not  as  an  isolated  phenomena,  but  an  interrelated  constituent 
process  operating  at  varying  levels  of  complexity-the  elements  of  which  acquire  significance  only  in  the  context  in 
which  they  are  used.  The  explanations  introduced  and  their  implications  should  be  carefully  evaluated  hy 
instructional  designers  and  researchers  for  further  research  into  the  more  effective  design  of  visualized  instruction. 

Note:  This  Presentation  was  based  upon  an  article  submitted  to  the  International  Journal  of  Instructional  Media. 
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Abstract 

Computer  conferencing  is  becoming  cm  increasingly  important  tool  in  university  course  delivery.  This  study 
addresses  ways  in  which  computer  conferencing  impacted  interaction  and  communication  patterns  among  students 
and  with  the  instructor  in  a graduate  course . Primarily  through  qualitative  research  methods,  we  address  the  use  of 
a widely  used  computer  conferencing  software  program  in  an  academic  setting.  We  also  identify  instructional  and 
learning  strategies  in  a constructivist  paradigm,  and  we  analyze  the  instructor's  role  in  computer  conferencing . 

Introduction 

Networked  computers  have  been  used  in  academic  course  delivery  in  the  United  States  since  as  early  as  the 
1980s,  and  their  use  grew  significantly  in  that  decade.  Hillz  (1994,  describes  a Virtual  Classroom™  as  a leaching 
and  learning  environment  located  within  a computer-mediated  system.  Rather  than  being  built  of  bricks  and  boards, 
it  consists  of  a set  of  group  communication  and  work  “space”  and  facilities,  which  are  constructed  in  software  (p.  3). 

A number  of  studies  have  examined  the  benefits  and  challenges  associated  with  this  educational  delivery 
system  (Berge,  1997;  Burge,  1994;  Harasim,  1990;  Hiltz,  1994).  Key  benefits  of  computer  conferencing  include 
providing  equal  access  for  interaction,  fostering  collaboration,  allowing  for  learner  reflection,  and  supporting  learner 
interaction.  Challenges  to  the  use  of  computer  conferencing  include  unequal  access  to  hardware  and  software,  a 
steep  learning  curve,  the  reliance  on  text-based  communication,  and  managing  large  amounts  of  information  and 
communication. 

Computer  conferencing  has  been  used  in  university  classes  as  an  adjunct  to  faec-to-facc  and  distant 
delivered  instruction  (Murphy,  Calhcarl,  & Kodali,  1997;  Yakimovicz  & Murphy,  1995)  as  well  as  the  primary 
mode  of  communication  (Eastmond,  1995;  Mclsaac  & Ralston,  1996).  These  university  sellings  provide  powerful 
evidence  of  the  ways  that  computer  conferencing  is  used  to  provide  opportunities  for  collaborative  learning  and 
sharing  of  multiple  perspectives,  both  of  which  arc  integral  to  constructivist  learning  environments  (Wilson,  1995). 

Of  the  many  computer  conferencing  software  programs  available,  FirstClass1M  (FC)  is  currently  the  most 
widely  used  (Bales,  1995).  The  FC  environment  provides  for  discussions  on  organized  topics,  uploading  and 
downloading  files,  real-time  text-based  chat  rooms,  conference  messages,  collaborative  document  writing  space,  and 
private  email.  All  of  these  functions  were  used  in  the  class  and  examined  in  this  study. 

This  study  addresses  the  manner  in  which  computer  conferencing  enhanced  communication  and  interaction 
patterns  among  graduate  students  in  a course  offered  at  a distance.  Wc  identify  the  capabilities  and  challenges  of 
computer  conferencing,  and  wc  address  the  similarities  and  differences  in  the  use  of  computer  communication  and 
face-to-face  interaction  strategies.  Wc  also  describe  how  the  medium  of  exchange  in  computer  conferencing 
encourages  online  discussion  and  fosters  collaborative  learning.  Wc  examined  the  instructor’s  expanded  role  in  the 
computer  conferencing  environment. 
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Theoretical  Framework 

Constructivism  is  the  paradigm  or  world  view  that  recognizes  learning  as  the  process  of  constructing 
meaning  about,  or  making  sense  of,  our  experiences.  Qualitative  researchers  “attempt  to  understand  the  meaning  of 
events  and  interactions  to  ordinary  people  in  particular  situations”  (Bogdan  & Biklcn,  1992,  p.  34).  Through 
working  in  collaborative  groups  (Brown  & Palincsar,  1989)  and  learning  in  authentic  environments  (Brown,  Collins, 
& Duguid,  1989),  learners  are  encouraged  to  develop  personal  meaning.  Computer  conferencing  is  increasingly  the 
means  by  which  “learners  actively  construct  knowledge  by  formulating  iacas  into  words  that  arc  shared  with  and 
built  upon  through  the  reactions  and  responses  of  others”  (Harasim  ct  al.,  1995,  p.  4). 

Garrison  (1989)  identifies  two-way  interaction  as  a critical  feature  of  the  educational  process.  In  a 
computer  conferencing  environment,  this  communication  takes  place  via  technology  and  is  embedded  in 
instructional  methods  that  provide  for  interaction  (Wagner,  1994).  Interaction  is  necessary  not  only  for  students  to 
receive  feedback  on  their  progress,  but  also  to  engage  learners  in  active  learning.  Computer  conferencing  should 
include  interactive  elements  that  require  learners  to  construct  meaning  actively  within  the  computer-based 
environment.  Research  indicates  that  higher  levels  of  interaction  typically  lead  to  more  positive  attitudes  toward  and 
greater  satisfaction  with  learning  (Hackman  & Walker,  1990).  Interaction  in  distance  education  typically  occurs 
between  the  learner  and  the  content,  the  learner  and  the  instructor,  and  the  learner  with  other  learners  (Moore,  1989). 
More  recently  it  has  been  noted  that  interaction  is  critical  between  learners  and  the  technology,  particularly  with 
high  technology  communication  devices  (Hillman,  Willis,  & Gunawardena,  1994). 

Researchers  have  reported  on  the  remarkable  contributions  of  computer  conferencing  to  collaborative 
learning  (Davie  & Wells,  1991;  Harasim  ct  al.,  1995;  Hiltz,  1994).  Collaborative  learning  and  computer 
conferencing  arc  reciprocally  related:  while  computer  conferencing  depends  on  the  ability  and  willingness  of 
participants  to  collaborate,  collaborative  learning  is  enabled  by  computer  conferencing  (Cifuentes,  Murphy,  Segur, 
& Kodali,  in  1998).  Romiszovvski  and  Mason  (1996)  posit  that  computer-mediated  communication  provides  for  two 
opposing  paradigms:  instructional,  or  traditional  education,  and  conversational,  which  occurs  in  collaborative 
learning  environments.  This  conversational  style  is  evident  in  “learning  environments  that  arc  more  authentic, 
situated,  interactive,  project-oriented,  interdisciplinary,  learner-centered”  (Berge,  1997,  p.  13). 


Overview 

In  a semester-long  university  course,  graduate  students  at  Texas  A&M  University  used  FC  software  from 
their  own  sites  or  a university  computer  lab  to  communicate  with  each  other  and  with  the  instructor  as  an  adjunct  to 
the  weekly  class  sessions  held  by  two-way  interactive  videoconference.  Specifically,  the  students  in  this  interactive, 
project-oriented  course  used  the  asynchronous  communication  features  of  the  text-based  FC  system  to  turn  in 
assignments,  moderate  conferences,  participate  in  other’s  conferences,  conduct  collaborative  writing  projects,  attach 
files,  and  conduct  real-time  discussions  through  the  synchronous  chat  mode.  The  students  accessed  FC  from 
centrally  located  computer  laboratories  on  the  university's  main  campus,  from  their  own  locations  via  modem  with  a 
PPP  or  SLIP  connection,  or  from  other  locations  with  a direct  connection  to  the  Internet.  With  the  exception  of  two 
occasions  on  which  technical  problems  prevented  access  to  the  server  from  outside  of  the  local  area,  students  had 
ongoing  access  to  FC. 

The  instructor  set  up  the  conferences  on  the  FC  desktop  throughout  the  semester.  Individual  icons  on  the 
desktop  represented  the  19  first-level,  asynchronous  conferences.  By  the  end  of  the  semester,  six  of  the  conferences 
contained  between  two  and  eight  sub-conferences  each.  In  only  one  conference  was  there  an  additional  level  of  sub- 
conferences  (see  Figure  1 for  a scanned  image  of  the  FC  desktop).  Although  the  FC  desktop  doesn’t  reflect  real-time 
chats,  the  students  used  live  chats  with  the  entire  class  on  the  three  occasions  that  they  did  not  meet  via  interactive 
videoconference.  Additionally,  as  they  became  more  familiar  with  the  chat  mode  during  the  semester,  students  and 
instructor  alike  chatted  on  informal  bases  with  one  or  two  others  at  a time. 
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Objective 

Patterns  of  communication  and  the  process  of  constructing  meaning  in  formal  education  can  be  profoundly 
affected  by  computer  conferencing,  particularly  in  university  courses  in  which  students  do  not  meet  facc-lo-face  on  a 
regular  basis.  To  determine  the  patterns  of  communication  in  computer  conferencing,  we  asked  the  following 
questions:  l)  What  arc  similarities  and  differences  in  interaction  and  collaboration  between  computer  conferencing 
and  face-lo-faec  learning  environments?  2)  How  does  the  medium  of  exchange  in  computer  conferencing  encourage 
or  discourage  online  discussion?  3)  What  collaborative  learning  strategics  arc  used  in  CC?  4)  What  is  the 
instructor’s  role  in  the  computer  conferencing  environment? 


Methods 

The  research  team  consisted  of  the  instructor  and  two  graduate  students,  one  from  the  course  that  was 
studied  and  another  from  a similar  course  during  the  same  semester.  The  three  researchers  collaborated  near  the  end 
of  the  semester  in  the  process  of  data  collection,  analysis,  writing,  and  rewriting. 


Subjects 

The  subjects  of  the  study  were  the  nine  registered  students  and  the  instructor  of  the  class.  The  students  had 
wide-ranging  initial  expertise  and  interest  in  telecommunications  in  a range  from  little  to  no  experience  with  e-mail 
prior  to  the  beginning  of  the  semester  to  telecommunication  professionals  who  work  in  the  field.  Adding  to  the 
complexity  of  communication  were  two  students  whoso  second  language  was  English. 

The  prc-coursc  surveys  revealed  that  more  than  half  of  the  nine  students  reported  having  minimal  or  no 
experience  at  all  with  distance  education  prior  to  the  beginning  of  the  semester.  The  majority  of  the  six  doctoral  and 
three  master’s  students  indicated  that  they  preferred  to  take  the  class  via  distance  technologies  and  fell  that  the 
course  would  hold  their  attention.  The  students  generally  anticipated  that  they  would  lake  more  responsibility  for 
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learning  in  this  class  than  in  a more  traditional  classroom  and  that  they  would  achieve  as  much  in  this  course  as  in  a 
course  taught  by  more  traditional  means.  They  each  expected  that  active  communication  and  interaction  with  the 
instructor  and  their  classmates  to  be  as  good  as  it  would  be  in  a traditional  class.  Each  expected  that  the  course 
would  help  them  learn  to  communicate  easily  with  students  in  other  locations.  Several  of  the  students  identified  their 
primary  reason  for  enrolling  in  this  elective  course  as  gaining  experience  in  distance  learning  technologies  and 
resources. 

Data  Collection  and  Analysis 

At  the  end  of  the  semester,  we  downloaded  and  printed  the  electronic  file  of  messages  from  all  of  the 
asynchronous  conferences  and  the  logs  of  the  live  chats.  The  data  sources  included  transcripts  of  all  computer 
conferences,  real-time  (synchronous)  chats,  and  students’  electronic  journals  about  computer  conferencing;  results 
of  pre-course  surveys  and  post-course  computer  conferencing  attitude  surveys;  and  notes  from  semi-structured 
interviews  with  participants. 

By  using  the  constant  comparative  method  (Bogdan  & Biklen,  1992),  we  discovered  from  the  student  data 
and  emergent  categories  during  the  analysis  process  that  we  needed  to  modify  our  research  questions.  This  method 
is  not  a slep-by-step  process.  Instead,  it  is  a dynamic  process  involving  all  actions:  collecting  data;  looking  for  key 
issues,  which  become  categories  of  focus:  looking  for  data  that  provide  many  incidents  of  the  categories  of  focus; 
writing  about  the  categories  being  explored;  and  attempting  to  describe  and  account  for  all  incidents  in  data  while 
continually  searching  for  new  incidents. 

In  this  study,  prior  to  data  analysis,  we  reviewed  the  literature  for  similar  studies  and  to  understand  data 
categories  identified  by  previous  research.  We  coded  and  categorized  the  data  from  the  transcripts  by  having  one 
researcher  go  through  the  data  and  evaluate  each  message  based  upon  the  type  of  information  or  communication 
contained  in  the  transcript.  The  two  other  researchers  then  reviewed,  discussed,  and  revised  the  coded  categories  as  a 
measure  of  coding  reliability.  For  example,  while  individually  reading  the  transcripts,  we  highlighted  and  assigned 
code  words  to  identify  the  types  of  interactions  or  communications  that  we  thought  were  taking  place.  Examples  are 
“response  to  peers,”  “strokes”  (verbal  affirmations),  and  “modeling  by  experience  sharing.”  We  then  looked  at  the 
codes  to  determine  natural  categories.  Most  of  the  code  words  occurred  many  times  in  the  data;  in  addition,  some  of 
the  codes  collapsed  into  larger  categories. 

We  established  the  following  codes  to  identify  quoted  messages,  which  are  written  precisely  as  ihc  authors 
wrote  them  in  the  conferences.  The  author  of  the  message  is  “I”  for  Instructor  and  “SI”  - “S9”  for  the  nine  students. 
The  four  types  of  conferences  that  emerged  from  the  data  were:  auxiliary  (outside  of  the  curriculum  of  the  class), 
instructional  (led  by  the  instructor),  student-moderated,  and  mctacognitivc  (shared  student  reflections).  Of  the  four 
types  of  conferences,  one  was  originally  student-moderated  and  later  became  mctacognitivc.  We  identified  the 
conferences  by  initials;  thus,  CMC  Communication  (student-moderated)  was  “CC-S while  CMC  Communication 
(metaeognitive)  was  “CC-M.”  The  date  of  each  message  was  also  included.  For  example,  a message  that  Student  5 
wrote  in  the  student-moderated  section  of  CMC  Communication  on  October  10,  1.996  would  be  identified  with  this 
description  following  the  direct  quotation  from  the  transcript:  [S5,  CC-S,  10/10/96].  The  emergent  categories  that 
were  identified  in  each  type  of  conference  are  listed  in  Table  1 below.  Three  of  the  four  instructional  conferences 
had  several  sub-conferences,  and  in  two  of  the  conferences  were  sub-conferences  arranged  for  small  groups  to 
discuss  issues  and  post  their  paced  assignments  for  each  other.  Figure  2 is  a flow  chart  of  the  four  types  of 
conferences  and  the  specific  conferences  that  fell  into  each  type. 
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Table  1 : The  Code  Words  Found  in  Iiach  Type  of  Conference 


Student  Moderated 

Conferences 

Instructional  Conferences 

Auxiliary  Conferences 

Metacognitive  Conferences 

Discussion  questions 
Response  to  peer 
Modeling  by  experience 
sharing 
Waving 
Peer  thank  you 
Closure 
Opinion 

Experience  sharing 
Response  to  moderator 
Technical  feedback 

Instructions 

Strokes 

Instructional  questions 
Response  to  instruction 
Opinions 
Project  update 
Response  to  peer 
Resource  sharing 
Peer  thank  you 
Technical  feedback 

Instructions 
Resource  sharing 
Technical  feedback 
Technical  questions 
Response  to  peer 
Peer  thank  you 
Information  sharing 

Perceptions 
Inslructio  1 strategics 
Impact 
Barriers 

Communication  strategics 
Response  to  instructor 

Figure  2:  Flow  Chart  of  the  Types  of  Asynchronous  Computer  Conferences 


Findings 

Following  arc  the  findings  related  to  the  research  questions.  A summary  of  the  types  of  messages  in  each  of 
the  four  conference  categories  is  reported  in  Figures  3,  4,  and  5.  These  three  figures  identify  the  types  of  interaction 
and  communication  that  took  place  among  the  participants.  Figure  3 reports  student-moderated  conferences.  The 
instructor  took  a minor  role  in  these  student-moderated  conferences.  The  largest  number  of  interactions  in  this 
conference  type  was  the  51  opinion  messages  posted  by  students.  The  next  highest  level  of  interactions  in  this 
category  was  student  responses  to  peers.  In  Figure  4,  instructional  conferences  are  listed.  These  conferences  were 
instructor-led  and  in  large  part  had  no  moderator.  The  largest  number  of  postings  in  these  conferences  was  the  43 
student  “Response  to  Instruction”  messages.  The  second  most  frequent  message  category  was  student  “Opinions.” 


301 


300 


Figure  5 lists  auxiliary  conferences,  which  were  conferences  established  for  interactions  that  were  outside  of  the 
curriculum  of  the  course.  Examples  arc  “Howdy”  for  students  to  get  acquainted  with  each  other  and  other  technical 
and  administrative  topics.  The  largest  number  of  interactions  in  this  category  was  student  “Resource  Sharing’’  with 
21  messages.  The  second  most  frequent  postings  were  “Response  to  Peer”  which  were  students  giving  feedback  to 
each  other. 


Figure  3.  Student  Moderated  Computer  Conferences 
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Figure  4:  Instructional  Conferences 

(Case  Studies,  Virtual  Meetings,  Class  Files,  Site  Visits,  Culture) 
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Because  the  mctacognitivc  conferences  consisted  of  individual  reflection  rather  than  reactions  to  others, 
Figure  6 summarizes  these  conferences  in  terms  of  the  types  of  reflections  that  the  participants  posted.  After  setting 
up  the  conferences,  posting  forms  (like  the  formative  evaluation  form),  and  requesting  feedback,  the  instructor 
played  an  insignificant  role  in  these  conferences  and  is  therefore  not  included  in  the  figure.  The  most  frequent  type 
of  interaction  in  this  conference  type  was  student  reflections  on  “Instructional  Strategies.” 

Figure  5.  Auxiliary  Computer  Conferences 

(Read  Me  First,  The  Internet,  Hardware  & Software,  Technical  /<£  A,  Mistakes,  Howdy) 
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In  addition  to  supplying  a quantitative  report  in  Figures  3,  4,  5,  and  6 on  the  ways  in  which  computer 
conference  participants  interacted  and  communicated  with  each  other,  we  address  the  research  questions  in  a more 
qualitative  manner.  Wc  do  so  by  looking  at  each  of  the  questions  in  turn  and  providing  evidence  through  verbatim 
statements  made  by  the  participants  themselves  in  conference.  Abbreviations,  spelling,  and  punctuation  arc  printed 
here  just  as  in  the  students’  original  contributions.  The  issues  discussed  are  related  to  comparisons  of 
communication  and  interaction  patterns,  encouraging  online  discussion,  collaborative  learning,  and  the  role  of  the 
instructor  and  obstacles  to  successful  computer  conferencing. 

Communication  and  Interaction  Patterns 

Online  interactions  share  many  characteristics  with  face-to-face  education:  input  of  ideas,  class  discussions, 
debates,  and  other  forms  of  knowledge  building  through  interaction  and  exchange  (Harasim,  1990).  We  identified  a 
number  of  barriers  and  benefits  of  communication  and  interaction  in  computer  conferencing. 

The  barriers  to  computer  conferencing  were  dramatic.  Students  experienced  an  initial  constraint  as  a result 
of  having  to  type  all  comments,  a process  that  "requires  an  adjustment  to  a new  form  of  communication  (i.e.,  using 
keyboard  to  communicate  with  short  statements,  learning  to  pause  for  feedback,  etc.)"  [SI,  CJ,  12/6/96].  They 
quickly  discovered  that  the  steep  learning  curve  was  greater  for  students  whose  first  language  is  not  English  and  for 
those  with  poor  typing  skills.  All  of  the  students  found  managing  large  amounts  of  information  and  communication 
to  he  a challenge. 

This  reliance  on  text-based  communication  provided  several  benefits,  however,  which  in  most  cases  were 
benefits  that  the  students  learned  through  the  process  of  computer  conferencing.  For  example,  students  learned  to 
verbalize  their  thoughts  textually  using  the  delay  in  the  asynchronous  discourse  to  reflect  upon  the  content  and  the 
issue  of  time: 
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Although  computer-mediated  conferencing  and  email  are  instant  delivery  media,  the  human  part  - thinking 
it  over  and  responding  still  takes  lime.  It  some  cases  it  can  take  longer  when  responding  to  email  or  a 
conference  electronically  than  doing  it  in  person.  We  tend  to  be  careful  of  what  wc  put  down  if  it  is  saved 
and  displayablc  for  others  to  sec.  [S6,  FE,  10/2/96] 

Because  the  immediate  response  time  found  in  face-to-face  environments  did  not  exist,  they  adapted  their 
textual  communications  to  become  more  descriptive,  detailed,  and  reflective.  Students  compared  their  own 
behaviors  in  computer  conferencing  with  those  in  facc-lo-face  environments,  recognizing  that  in  the  computer 
conferencing  environment  “off-task”  behaviors  arc  more  obvious, 

I remember  making  some  of  those  off  task  comments  during  the  chat.  When  you  think  about  it  those  side 
comments  arc  the  equivalent  to  speaking  out  in  class  interrupting  another  conversation  with  a totally  out  of 
place  comment...  However  there  seems  to  the  abih.y  to  ignore  some  of  this  because  you  can  always  see 
what  the  people  on-task  arc  saying  and  document  what  the  off-task  people  are  doing  :)  [S2,  CJ,  1 1/23/96]. 
Such  comparisons  of  communication  and  interaction  in  computer  conferencing  and  face-to-face  instructional 
environments  indicate  that  when  encouraged  to  do  so,  students  become  aware  of  communication  patterns  that  take 
place  in  their  conferences  and  even  become  critical  of  their  own  behaviors 

Encouragement  of  Online  Discussion 

The  medium  of  exchange  in  computer  conferencing  tends  to  encourage  discussion  by  allowing  participants 
to  learn  at  their  own  pace  and  reflect  before  replying  to  a message.  Live  chats  were  observed  to  help  draw  out 
students  who  were  traditionally  quiet  in  the  face-to-face  classroom  environment.  Once  students  overcame  obstacles 
like  inadequate  access  to  hardware  and  software,  technical  problems,  and  discomfort  with  new  telecommunications 
software,  they  remarked  that  computer  conferencing  allowed  them  to  have  discussions  that  fostered  learning,  and 
they  suggested  strategies  for  moderating  effective  conferences  and  online  chats.  Barriers  to  online  discussion 
included  the  fact  that  the  speed  of  typing  is  slower  than  verbal  interaction,  and  that  threads  of  the  discussion  can 
become  confusing  if  not  actively  managed.  The  advantages  of  this  form  of  communication  included:  students  had 
more  time  to  read,  reflect,  write,  and  revise  their  ideas;  traditionally  quiet  students  tended  to  increase  expression  and 
communication;  students  were  provided  with  an  immediately  relevant  mechanism  for  sharing  strategics  for  effective 
conferencing;  and  the  communication  of  the  formal  classroom  meetings  was  enhanced  by  augmenting  discussions 
and  exchange  of  ideas. 

Collaborative  Learning  Strategies 

Researchers  suggest  that  the  very  nature  of  computer  conferencing — its  capacity  to  support  interaction 
between  and  among  students  and  teachers — fosters  a collaborative  approach  to  learning.  Collaborative  learning 
refers  to  “any  activity  in  which  two  or  more  people  work  together  to  create  meaning,  explore  a topic,  or  improve 
skills”  (Harasim  ct  al.t  1995,  p.  30).  In  an  educational  environment,  collaborative  learning  "means  that  both  teachers 
and  learners  are  active  participants  in  the  learning  process;  knowledge  is  not  something  that  is  'delivered'  to  students, 
but  rather  something  that  emerges  from  active  dialogue  among  those  who  seek  to  understand  and  apply  concepts  and 
techniques"  (Hiltz,  1994,  p.  23). 

Barriers  to  collaborative  learning  included  differences  in  team  member  contributions  and  variability  in 
access  to  computer  equipment.  As  adult  learners,  each  of  the  graduate  students  balanced  outside  work,  family,  and 
schedules  that  impacted  their  timing  and  frequency  of  participation.  The  advantage  of  using  computer  conferencing 
with  these  adult  learners  is  that  they  were  able  to  participate  in  collaborative  learning  projects  at  the  time  and  place 
most  convenient  for  them.  Some  students  accessed  FC  from  remote  locations  and  some  used  equipment  located  on 
the  campus.  Remote  students  generally  experienced  greater  barriers  to  access  to  computer  equipment  than  did  on- 
site students. 

Advantages  to  computer  conferencing  arc  that  it  supports  instruction  by  allowing  for  communication 
outside  of  class  time,  and  it  fosters  collaborative  learning  by  providing  time  and  place  independent  communication 
vehicles  for  instructor  and  students.  Students  commented  that  the  collaborative  structure  of  the  conferences  helped 
them  advance  academically  and  made  them  feel  part  of  a larger  group.  An  example  of  one  collaborative  learning 
strategy  concerned  the  student’s  discussion  of  the  problems  and  frustrations  related  to  managing  the  many  sources  of 
information  including  email,  voice  mail,  paper  mail,  beepers,  and  other  electronic  sources  of  messages  and 
interruptions.  Students  shared  intellectual,  practical,  and  emotional  strategics  for  dealing  with  information  overload 
in  our  modern  technological  environment.  While  weaving  a conference,  a student  moderator  wrote  that  his 
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classmates  had  contributed  a number  of  good  ideas  and  strategies  for  dealing  with  the  volume  of  information  coming 
from  today’s  technologies. 

Instructor’s  Role  in  Computer  Conferencing 

The  instructor’s  role  in  a computer  conferencing  environment  tends  to  be  different  from  facc-to-face 
instructional  settings.  Through  CC,  an  instructor  may  provide  guidance  or  communication  to  students  privately, 
without  drawing  attention  to  the  action  as  in  a facc-to-face  classroom  by  taking  students  aside  or  asking  them  to  stay 
after  class.  An  instructor  may  also  more  equilaoly  post  equal-access  announcements  to  students  outside  of  the 
classroom  environment  through  email  distribution  to  the  enrolled  students.  Students  recognized  that  prompt 
feedback  on  assignments  is  an  advantage  of  online  class  conferences.  Additionally,  advance  planning  and  a clear 
structure  help  the  students  concentrate  on  the  content  of  real-time  chats. 

The  instructor’s  role  in  a computer  conferencing  environment  is  different  from  facc-to-face  instructional 
settings  (Gunawardena,  1992).  In  computer  conferencing,  the  instructor  must  spend  significantly  more  time  on  a 
more  frequent  basis  responding  to  the  constant  evolution  of  the  online  processes  (Weiss  & Morrison,  1998).  One 
student  remarked  in  the  discussion  section  of  an  instructional  conference  that  prompt  feedback  on  assignments  is  an 
advantage  of  online  class  conferences: 

For  those  of  us  at  distant  sites,  sending  in  hard-copy  papers  and  waiting  for  them  to  be  mailed  back  in  the 
“traditional  mode”  is  often  a process  that  takes  weeks  - in  the  meantime  additional  assignments  are  done  without 
feedback  from  the  first.  By  putting  our  work  in  FirstClass,  we  have  the  benefit  of  feedback  before  any  more 
assignments  arc  due  - and  using  all  recycled  electrons!  [S9.  SV-D,  9/16/96].  Similarly,  advance  planning  and  a clear 
structure  help  the  students  concentrate  on  the  content  of  their  conferences.  One  student  remarked  in  the  final  entry  of 
his  reflective  journal  (which  the  students  kept  in  FC  throughout  the  semester): 

Dr.  M’s  arrangement  for  group  and  individual  time,  with  specific  instructions  on  how  each  is  to 
be  carried  out,  really  helps  to  keep  the  computer  conferencing  focused  and  moving  forward  while 
encouraging  everyone  to  participate.  [SI , CJ,  12/6/96]. 

Those  who  have  incorporated  computer  conferencing  into  university  courses  recognize  that  an  immense  amount  of 
advance  planning,  structure,  and  provision  for  training  on  hardware  and  software  are  necessary  for  the 
communication  and  interaction  to  be  beneficial  to  learners  (Gunawardena,  1992;  Mclsaac  & Ralston,  1996;  Murphy, 
Cathcart,  & Kodali,  1997).  This  study  found  that  in  addition  to  giving  prompt  feedback,  providing  advance  planning 
and  clear  structure,  and  planning  for  hardware  and  software  training,  the  instructor  must  play  a variety  of  roles 
including  those  of  facilitator,  coach,  guide,  expert  resource,  and  arbitrator. 


Conclusions  and  Relevance  to  the  Field 

Computer  conferencing  is  an  information  tool,  a communication  tool,  and  a generative  tool  that  fosters 
creativity  and  problem  solving  (Jonassen,  1996).  It  offers  an  effective  and  efficient  means  of  providing  information, 
generating  ideas,  and  communicating  to  the  users.  Easy-to-use  software  features  allow  the  user  to  concentrate  on  the 
content  rather  than  the  mechanics  of  the  computer  conferencing  environment.  Computer  conferencing  supports 
active,  self-directed  learning  using  a structure  that  is  easily  modified  to  fit  the  different  needs  of  learners  and  the 
instructor.  Sharing  ideas,  discussing  experiences,  and  clarifying  concepts  in  the  synchronous  and  asynchronous 
computer  conferencing  environment  promotes  interaction  among  the  students  and  the  instructor.  These  interactions 
encourage  discussion  and  reflection,  provide  accessibility  at  any  time,  and  stimulate  critical  thinking,  application, 
and  synthesis. 

Current  periodicals,  listservs,  and  scholarly  research  have  focused  increased  attention  upon  the  use  of 
computer  conferencing  to  enhance  learning  (cf.,  Yakimovicz  & Murphy,  1995).  Computer  conferencing  is  an 
important  educational  tool  because  of  its  effect  on  learning  processes,  changing  rules  of  communication,  and  impact 
on  how  students  enhance  their  knowledge.  If  knowledge  is  defined  as  the  capacity  for  effective  action,  computer 
conferencing  is  a rich  learning  environment  for  student  enhancement  of  knowledge  through  action. 
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Introduction 

Support  for  Teachers  Enhancing  Performance  in  Schools  (STEPS)  is  a World  Wide  Web  and  CD-ROM 
EPSS  designed  for  PrcK-12  and  prcservicc  educators  to  assist  in  designing  and  developing  lessons,  units,  or 
curricula.  This  “just-in-time”  tool  was  designed  around  school  reform  and  accountability  initiatives  in  the  state  of 
Florida.  School  districts  arc  required  to  align  existing  district-level  standards  to  Florida’s  new  Sunshine  State 
Standards  (which  is  Florida’s  response  to  Goals  2000).  Educators  are  required  to  implement  the  Sunshine  State 
Standards  in  all  subject  areas.  Districts  will  be  held  accountable  for  student  outcomes  related  to  the  Sunshine  State 
Standards  through  a new  test,  the  Florida  Comprehensive  Achievement  Test  (FCAT)  beginning  in  1999. 

To  educate  both  preservicc  and  inscrvicc  educators  on  the  new  Sunshine  State  Standards,  massive  staff 
development  is  required  across  the  state.  Rather  than  conducting  exhausting  hours  of  inservice  training  where 
hundreds  attend,  receives  a notebook,  and  promptly  returns  to  the  classroom  to  continue  with  business  as  usual,  it 
was  concluded  that  a virtual  approach  to  training  and  continuous  support  would  be  more  effective  and  reach  the 
critical  mass  of  educators  in  a shorter  time  frame.  With  the  goal  of  training  teachers  on  Florida’s  Sunshine  State 
Standards,  it  became  apparent  that  an  electronic  performance  support  system  (EPSS)  could  serve  as  an  innovative 
approach  to  staff  development  and  ongoing  support.  The  anticipated  outcomes  include  day-one  performance  of 
instructional  planning  using  the  Sunshine  State  Standards;  ongoing  support  through  scaffolding  instructional 
tutorials  and  guidance  within  STEPS;  and  for  idea  sharing  among  educators  across  the  state  with  all  outcomes 
geared  around  the  four  primary  areas  of  accountability. 

Areas  of  Accountability 

Instructional  planning  is  geared  around  four  areas  of  accountability  common  in  restructuring  efforts:  (1) 
integrated  curriculum;  (2)  integrating  technology;  (3)  alternative  assessment;  and  (4)  diverse  learning  environments. 
For  educators  to  weave  the  lour  areas  of  accountability  into  the  instructional  planning  process,  it  was  critical  that 
teachers  understand  more  clearly  what  each  area  means  within  the  individual  classroom  setting  as  well  as  in  the 
larger  school  and  district-level  environment. 

Integrated  curriculum  is  defined  differently  by  elementary;  middle  and  high  school  teachers.  Elementary 
teachers  view  integrated  curriculum  as  a common  strategy  for  instructional  delivery.  Elementary  teachers  typically 
plan  for  thematic  units  across  curriculum  areas  and  teach  the  same  kids  throughout  the  day.  Middle  and  high  school 
teachers  have  a more  difficult  time  adjusting  to  this  approach  simply  due  to  the  “departmental”  structure  within  the 
school  environment.  The  mathematics  teacher  teaches  mathematics  and  in  the  past  has  not  considered  how  to  weave 
other  subject  areas  into  his  or  her  classroom.  Nor  has  it  been  contemplated  how  to  team  teach  with  teachers  of  other 
subject  areas.  The  complications  continue  when  trying  to  plan  for  the  team  approach  when  teachers  do  not  have 
common  planning  time  or  when  teachers  do  not  serve  the  same  students. 

Integrating  technology  is  a restructuring  effort  that  most  teachers  agree  should  be  accomplished;  however, 
pitfalls  arise  when  software  and  hardware  arc  not  available;  when  teachers  arc  not  trained;  and  when  teachers  do  not 
have  the  time  required  to  learn  new  technologies  (U.S.  Congress,  1986).  With  all  of  the  barriers  that  exist,  many 
teachers  do  not  achieve  true  technology  integration  within  the  classroom  environment  for  several  years.  It  may 
never  be  achieved  without  district  and  building-level  administrator  support  (Northrup  & Little,  1986). 

Alternative  assessment  is  used  very  loosely  within  the  school  environment  without  much  consideration  to 
instructional  alignment  of  objectives,  content,  and  assessment  strategy.  Using  various  integrated  curriculum  models 
such  as  problem-based  or  projcct-bascd  learning  further  complicate  an  educators  understanding  of  how  to  assess. 
Many  teachers  consider  the  data  collection  strategy  such  as  using  a portfolio  the  solution  to  alternative  assessment. 
However,  many  teachers  do  not  further  define  the  data  elements  that  should  be  included  within  the  portfolio  nc  the 
assessment  criteria  that  should  be  used  to  judge  the  quality  of  the  work. 
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Diverse  learning  environments  can  be  defined  by  educators  in  two  general  categories:  (1)  the  learning 
environment  and  (2)  the  learner.  Within  the  learning  environment,  a teacher  may  choose  to  use  cooperative  learning 
as  a grouping  strategy  or  may  choose  to  integrate  instructional  technology  into  the  classroom.  When  examining  the 
individual  learner,  multiple  intelligences,  learning  styles,  and  diversity  arc  considered.  Many  veteran  teachers 
intuitively  adjust  teaching  style  and  strategies  based  on  the  composition  of  learners  in  the  classroom.  Preservice 
teachers  do  not  have  the  prior  knowledge  of  how  students  learn  to  intuitively  make  decisions  regarding  the  learning 
environment  and  the  learner.  A formal  system  of  analyzing  the  learner  and  the  environment  has  not  been  embedded 
in  teacher  planning  strategies,  nor  are  many  educators  aware  of  the  subtle  differences  among  learners  and  strategies 
that  can  be  used  to  meet  their  needs. 

Critical  Design  Variables 

To  achieve  the  four  areas  of  school  reform  and  accountability  and  to  maintain  that  all  teachers  would  be 
able  to  use  STEPS  for  instructional  planning  and  ongoing  support,  several  design  variables  were  established  as 
guiding  principles  for  the  development  of  STEPS.  Variables  include: 

Easy  access.  STEPS  was  designed  as  a World  Wide  Web  and  CD-ROM  tool,  ft  was  a given  that  all 
teachers  do  not  have  Internet  access  in  the  classroom.  However,  the  team  decision  was  not  to  design  for  the  past  but 
for  the  current-day  classroom  environment  and  the  future.  Most  teachers  have  access  to  a CD-ROM.  By  delivering 
STEPS  using  both  formats,  it  was  determined  that  most  teachers  can  gain  access  to  STEPS  either  in  the  classroom  or 
at  home. 

User  interface.  STEPS  was  designed  with  the  novice  Internet  and  CD-ROM  user  in  mind.  The  interface  is 
simple;  free  of  extraneous  information;  and  free  of  unnecessary  graphics,  animations,  and  videos.  Consideration  for 
download  time  via  the  Internet  required  forethought  on  message  design  decisions.  STEPS  has  a button  bar 
extending  the  entire  left  column  of  the  screen  with  text  and  graphic  buttons  for  user  navigation.  STEPS  provides 
directions  within  the  program  along  with  a Coach  for  scaffolded  navigational  assistance  as  required  by  the  user.  The 
theme  of  STEPS  varies  by  grade  level  cluster  and  is  followed  throughout  the  individual  components. 

Model  for  teacher  planning.  STEPS  must  be  a model  for  teacher  planning  geared  around  the  four  areas  of 
school  reform  and  accountability.  Using  the  Events  of  Instruction  (Gagne,  Briggs,  and  Wager,  1992)  as  the  guide, 
teachers  were  involved  in  the  design  of  the  model  for  planning.  Used  within  the  model,  are  “teacher  words”  as 
opposed  to  the  nine  events  of  instruction  as  stated  throughout  our  field.  The  Events  of  Instruction  is  the  underlying 
framework  with  teacher-driven  terminology  overlaying.  We  found  this  to  be  a significant  step  to  making  the  tool 
usable.  Additionally,  we  found  that  the  model  for  teacher  planning  would  be  somewhat  different  based  on  the  grade 
level  cluster  and  the  integrated  curriculum  approach  (thematic,  problem-based,  and  project-based  learning). 

Scaffolding  for  guidance.  Given  that  STEPS  is  a tool  used  by  teachers  in  their  own  environments,  it  is 
critical  to  provide  the  support  that  all  educators  may  require.  Scaffolding  is  a logical  design  strategy.  Educators 
using  STEPS  may  opt  for  additional  information  through  instructional  tutorials  or  for  additional  navigational 
assistance.  It  is  assumed  that  STEPS  will  be  used  for  ongoing  staff  development  at  that  point  of  need.  STEPS  is 
designed  for  teachers  to  use  over  and  over  again.  It  is  assumed  that  the  scaffolding  will  become  less  necessary  as 
teachers  gain  familiarity  with  the  tool. 

Materials  within  STEPS  arc  teacher  developed  and  tested.  All  materials  within  STEPS  arc  designed  by 
practicing  educators  who  know  what  works  in  the  classroom.  Teachers  were  contracted  to  work  on  the  project  and 
began  development  after  learning  about  the  Sunshine  Stale  Standards  and  understanding  the  four  primary  areas  of 
school  reform  and  accountability.  Teachers  developed  unit  plans,  best  practices,  and  identified  web  sites  that  would 
work  in  their  classrooms.  All  materials  have  been  reviewed  by  a quality  assurance  team  and  have  been  extensively 
field  tested  by  practicing  teachers  and  preservice  teachers. 

Plan  for  continuous  evaluation  and  revision  using  concurrent  design  principles.  STEPS  has  been  designed 
using  a variety  of  models  including  rapid  prototyping  (Jones,  Li,  & Merrill,  1992);  prototyping  (Tripp  & 
Bichclmcyer,  1990);  and  Layers  of  Necessity  (Tcssmer  & Wcdman,  1990).  The  concept  of  concurrent  design  (Witt 
& Wager,  1994)  and  evaluation  (Northrup,  1995)  have  permeated  the  process.  In  designing  an  EPSS,  initial  goals 
can  be  established,  however,  the  user  interface  is  the  key  to  successful  utilization  and  must  be  tested  and  revised 
throughout  the  development  process.  Additionally,  given  that  all  of  the  components  within  STEPS  arc  inputs  and 
outputs  of  the  entire  system,  continuous  evaluation  and  revision  are  essential. 
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Components  within  STEPS 

To  determine  the  components  within  STEPS,  attention  was  placed  on  the  overarching  definition  of  an 
EPSS  and  how  the  EPSS,  STEPS  should  be  structured.  Gcry  (1991)  defines  an  EPSS  as  an  integrated  environment 
that  contains  a full  range  v ' information,  software,  guidance,  advice  and  assistance,  data,  images,  tools,  and 
assessment  and  monitoring  systems.  The  key  component  of  Gcry’s  definition  is  that  an  EPSS  will  permit  job 
performance  with  minimal  support  and  intervention  by  others.  Raybould  (1995)  further  suggests  that  the  outcome  of 
an  EPSS  is  to  “enable  individuals  to  achieve  required  levels  of  performance  in  the  fastest  possible  time  and  with  a 
minimum  of  support  from  other  people."  Sherry  and  Wilson  (1996)  suggest  that  the  potential  of  an  EPSS  to  assist 
employees  in  gaining  job-related  skills  arc  overwhelming.  Employee  skills  arc  learned  by  doing,  not  by  being 
taught.  The  result  is  that  a relevant  structure  must  be  user-defined  and  the  system  must  be  flexible  to  accommodale 
for  multiple  uses  of  the  EPSS  system. 

The  conceptual  framework  established  by  Sherry  and  Wilson  that  employees  learn  by  doing  not  by  being 
taught  and  that  the  structure  be  user-designed  and  flexible  served  as  the  basis  of  STEPS.  Recall  that  the  purpose  of 
STEPS  is  to  assist  preservice  and  practicing  teachers  in  planning  using  Florida’s  Sunshine  State  Standards. 

Based  on  the  conceptual  framework  of:  (1)  flexibility;  (2)  learning  by  doing;  and  (3)  a user-designed 
structure,  while  maintaining  a clear  vision  of  the  purpose  of  STEPS,  several  component  parts  within  STEPS  were 
established.  To  guide  the  user  through  the  instructional  planning  process,  a Lesson  Architect  was  created.  The 
Lesson  Architect  is  what  Gery  (1991)  would  label  the  Infobasc.  It  includes  information,  tools,  and  the  methodology 
for  instructional  planning.  The  information  resources  are  layered  so  that  users  can  pursue  information  in  more 
depth,  just-in-time  and  at  the  point  of  need.  The  support  system  or  resources  that  interface  with  the  Lesson  Architect 
include:  model  units;  a best  practices  database;  web  links  to  over  400  web  sites;  links  to  the  Sunshine  Slate 
Standards  and  to  accountability  information;  and  over  40  instructional  tutorials.  The  user  interface  includes 
scaffolding  for  support  and  layered  inforination/rcsources.  The  final  layer  of  STEPS  is  an  InfoMAP  that  maps  the 
instructional  planning  process  for  educators  (see  Figure  1 for  a Diagram  of  STEPS).  A detailed  description  of  each 
component  follows: 

The  Lesson  Architect 

The  Lesson  Architect  is  the  ccntcrpiccc  of  this  EPSS  tool.  Users  of  STEPS  » \\  be  guided  to  the  Lesson 
Architect  to  begin  the  instructional  planning  process.  Each  component  of  the  process  addresses  one  of  the  four  areas 
common  in  restructuring  efforts,  requiring  users  to  contemplate  how  each  component  will  be  designed  into 
individual  lessons  and  activities.  The  theoretical  underpinnings  of  the  Lesson  Architect  arc  in  Gagne’s  Events  of 
Instruction  (1992),  the  Dick  and  Carey  model  (1996),  and  through  various  curriculum  approaches  including  webbing 
and  threaded  curriculum. 

To  use  the  Lesson  Architect,  users  will  insert  information  directly  into  forms  provided  on  the  world  wide 
web  page.  Embedded  within  the  Lesson  Architect  is  scaffolded  help  and  more  in-depth  information  on  each 
instructional  planning  element.  Users  can  access  the  best  practices  database;  the  mode!  units;  or  the  web  links  to 
gather  ideas  for  planning  the  unit  or  lesson.  Additionally,  instructional  tutorials  that  will  provide  instruction  on  over 
40  topics  are  embedded  within  the  Architect.  For  example,  one  of  the  categories  within  the  Architect  is  to  write 
instructional  objectives.  A link  to  a tutorial  that  instructs  users  how  to  write  an  objective  is  included.  Users  can 
select  to  link  to  the  tutorial  if  needed.  Additionally,  scaffolded  help  is  available  through  another  link  that  will 
describe  how  to  enter  information  into  the  form  or  how  to  copy  and  paste  information  from  another  location  on  the 
web  or  within  STEPS.  Finally,  a scaffolded  link  back  to  the  InfoMAP  will  provide  a concept  map  for  users  to 
determine  where  they  are  in  the  instructional  planning  process.  When  the  Lesson  Architect  is  complete,  information 
can  be  submitted  by  selecting  the  submit  button,  the  user  can  obtain  a printout  of  the  completed  lesson  plan,  or  can 
forward  it  on  to  the  principal,  the  practicum  teacher,  or  the  COE  instructor  for  review  through  email. 

Tutorial  Library 

The  Tutorial  Librarv  is  a collection  of  40  instructional  tutorials  that  arc  centered  around  the  four  primary 
areas  of  focus:  (1)  integrated  curriculum;  (2)  integrated  technology;  (3)  alternative  assessment;  and  (4)  diverse 
learning  environments.  The  purpose  of  the  Tutorial  Library  is  to  provide  just-in-time  support  to  users  who  arc 
attempting  to  plan  using  the  Lesson  Architect.  For  example,  if  a user  wants  to  integrate  email  into  an  instructional 
lesson,  but  is  unsure  how  to  actually  send  and  receive  email  messages,  a tutorial  on  email  is  one  click  away. 
Tutorials  are  designed  as  5-10  minute  overviews  with  opportunity  for  users  to  investigate  further  through  hypertext 
links  to  examples,  resources,  and  more  information  on  the  specific  topic  of  interest. 
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The  Model  Units 

Four  model  units  exist  currently  on  STEPS.  One  model  is  designed  for  each  of  the  four  grade  level 
clusters:  PK-2;  3-5;  6-8;  and  9-12.  Each  model  unit  was  created  by  teams  of  teachers  representing  the  grade  level 
clusters.  For  grades  PK-2,  Communities  is  the  theme  with  a 10-day  model  unit  accompanying.  Grades  3-5’s  unit  is 
on  16th  Century  Florida  History  as  students  go  online  and  search  for  missing  Luna  Ships  in  the  Pensacola  Bay. 
Grades  6-8  uses  Native  Florida  Habitats  as  the  theme.  Finally,  grades  9-12  uses  Terrestrial  Archaeology  is  the 
theme.  All  units  were  designed,  developed,  and  implemented  by  cross-curricular  teams  of  elementary,  middle,  and 
high  school  teachers.  Each  unit  is  comprised  of  10  days  of  instruction  designed  using  the  Lesson  Architect  that 
models  integrated  curriculum,  integrated  technology,  alternative  assessment,  and  diverse  learning  environments. 
The  mode!  curriculum  units  make  cross-curricular  connections  in  Math,  Science,  Social  Studies,  and  Language  Arts 
using  benchmarks  established  in  Florida's  Sunshine  State  Standards.  Within  the  model  units,  there  are  embedded 
hypertext  links  to  instructional  tutorials,  to  relevant  web  sites,  and  to  Florida’s  Sunshine  State  Standards. 
Additionally,  video  clips  modeling  various  aspects  of  the  unit  including  diverse  settings,  the  use  of  technology,  and 
others  are  available  within  the  10  day  units. 

The  model  units  serve  as  a focal  point  for  the  themes  within  each  grade  level  cluster.  The  units  arc 
designed  for  teachers  to  use  as  models  for  planning  their  own  units  using  Florida’s  Sunshine  Stale  Standards.  The 
units  link  to  four  online  Internet  expeditions  that  will  be  online  beginning  late  Spring  1998.  The  expeditions  enable 
teachers  to  fully  integrate  instructional  technology  in  the  classroom  while  following  exciting  expeditions  by 
archaeologists,  scientists,  and  community  workers,  (sec  http://www. uwf.edu/-pacce/stcps  to  access  the  online 
expeditions). 

Related  Web  Links 

Over  400  web  sites  have  been  identified  as  relevant  to  Math,  Science,  Social  Studies,  and  Language  Arts  to 
parallel  Florida’s  Sunshine  Stale  Standards.  Web  sites  arc  keyed  to  the  Sunshine  State  Standards  and  provide  a one- 
paragraph  summary  of  the  content  addressed  on  the  site.  All  sites  have  been  evaluated  for  instructional  integrity  and 
can  be  used  as  a resource  for  teacher  planning  using  the  Lesson  Architect  and  for  classroom  instructional  use. 

Best  Practices  Database 

The  Best  Practices  Database  is  a sharing  success  database  of  ideas  that  work  in  the  classroom.  Classroom 
teachers  from  several  school  districts  have  contributed  to  the  best  practices  database  by  submitting  lesson  plans, 
ideas  for  classroom  assessment,  technology  integration,  cooperative  learning,  and  more.  Within  STEPS,  users  can 
access  the  best  practices  database  cither  by  Sunshine  State  Standard  and  subject  area  or  by  a keyword  search.  Either 
approach  will  yield  significant  strategies  and  tactics  that  work  in  the  classroom.  It  is  intended  that  users  of  STEPS 
will  access  the  Best  Practices  Database  to  gather  ideas  for  instructional  planning  within  the  Lesson  Architect. 

The  Coach 

The  Coach  is  the  STEPS  approach  for  scaffolded  help.  It  is  designed  on  three  levels  to  provide  scaffolded 
support  throughout  the  entire  program.  Using  the  Knowledge  Integration  Environment  presently  being  created  by 
University  of  California  - Berkeley  as  the  model  (Bell,  Davis,  & Linn,  1997),  wc  have  established  a parallel 
‘‘Knowledge  Integration  Coach”  that  will  provide  help  on:  (a)  The  Big  Picture;  (b)  What  Do  I Do?;  and  (c)  How  Do 
I Do  It?  Each  level  of  scaffolded  help  will  serve  as  prompts  and  context-sensitive  feedback  as  students  work 
through  STEPS. 

Overall  Benefits  of  STEPS 

STEPS  is  an  EPSS  designed  to  assist  prcscrvicc  and  veteran  teachers  in  planning  using  Florida’s  Sunshine 
State  Standards.  Currently  STEPS  is  being  used  by  teachers  throughout  the  state  with  great  success.  Practicing 
teachers  welcome  the  just-in-time  approach  to  learning  how  to  plan  using  the  Sunshine  State  Standards.  Prescrvicc 
teachers  using  STEPS  are  being  prepared  to  enter  the  workforce  with  the  most  current  knowledge,  skills,  and 
abilities  in  school  reform  and  accountability.  Both  prescrvicc  and  practicing  teachers  have  a model  for  planning  that 
includes  support  and  guidance  along  with  iiterally  thousands  of  resources  and  ideas  for  classroom  implementation. 
Teachers  continue  to  report  that  resources  available  at  their  fingertips  is  the  most  significant  benefit  to  them. 

Though  unanticipated  in  the  initial  design  of  STEPS,  many  other  benefits  have  occurred.  Individual  school 
districts  are  requesting  that  STEPS  be  customized  to  meet  individual  district  needs  with  lesson  planning  and  with 
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best  practices.  Additionally,  districts  view  STEPS  as  an  optimal  tool  for  virtual  staff  development  within  school  and 
district  staff  development  plans.  It  has  been  requested  that  we  incorporate  pre  and  post  tests  for  instructional 
tutorials  and  for  the  Lesson  Architect  so  that  individual  teachers  may  receive  inservicc  points  for  completing 
components  within  STEPS. 

Both  preservice  and  practicing  teachers  and  school  districts  arc  using  STEPS  in  ways  not  originally 
anticipated.  The  conceptual  framework  of  building  a model  that  is  flexible  has  not  only  gained  benefits  for  the 
original  intent,  many  view  STEPS  as  an  iterative  process  for  continued  design  and  development  as  a “one-stop 
shopping”  too]  for  educators. 

Conclusion 

STEPS  is  an  electronic  performance  support  system  currently  being  used  by  middle  school  educators  in 
Florida.  It’s  capability  will  be  expanded  to  elementary  and  high  school  teachers  in  phase  II  of  our  g-anl,  beginning 
Summer  1997.  Additionally,  STEPS  is  being  used  in  Preservice  Teacher  Preparation  courses  lo  prepare  teachers  to 
integrate  curriculum  Sc  technology,  conduct  alternative  assessment,  and  work  in  diverse  learning  environments.  The 
components  of  STEPS  work  in  tandem  to  provide  just-in-iimc  training  and  support  lo  teachers  through  the  Lesson 
Architect  and  all  of  the  corresponding  materials.  Finally,  scaffolded  help  is  provided  to  guide  and  facilitate  the 
tcu.her  through  the  instructional  planning  process  using  a nurturing,  supportive  approach  with  video  and  audio  clips 
as  a guide. 
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Figure  1:  STEPS  Diagram 
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Abstract 

This  study  examined  critical  mass  for  the  use  of  information  technologies  (electronic  mail  and  Web 
applications)  in  Schools,  Colleges . and  Departments  of  Education  (SCDEs)  by  faculty,  students , and  faculty  and 
students  combined.  The  study  also  investigated  critical  mass  for  the  existence  of  adequate  infrastructure  for 
information  technologies.  A sample  survey  design  was  used  to  collect  data  from  professional  teacher  education 
institutions.  The  1996  SCDE  Technology  Survey  was  mailed  to  744  SCDE  institutions.  Responses  were  received 
from  465  institutions  for  a response  rate  of  65%.  Achievement  of  critical  mass  was  based  on  the  number  of 
institutions  identified  as  SCDE  faculty  and  student  users  (combined),  SCDE  faculty  users,  SCDE  student  users,  and 
SCDE  institutions  which  provide  adequate  infrastructure  for  information  technologies.  Results  confirm  critical  mass 
has  been  reached  in  SCDE  use  of  information  technologies  and  for  (he  provision  of  adequate  infrastructure  for  the 
use  of  information  technologies.  Recommendations  and  implications  for  other  issues  associated  with  technology 
integration  are  included. 

Introduction 

The  purpose  of  this  research  study  was  to  advance  the  knowledge  base  in  the  area  of  critical  mass  for  the 
use  of  and  infrastructure  for  information  technologies  (e-mail  and  Web  applications)  within  pre-service  teacher 
education  institutions.  While  the  results  of  this  study  arc  not  particularly  surprising  to  those  who  study  diffusion  and 
change  related  to  technology,  the  documentation  and  dissemination  of  the  results  have  strong  implications  for 
several  areas  associated  with  technology  integration  (c.g.,  accreditation  guidelines,  professional  teacher  standards, 
school  reform,  public  perception  of  teacher  preparation  in  and  access  to  technology).  This  study  was  developed  on 
the  functional  definition  of  critical  mass  given  by  Rogers  (1995,  p.  313):  “the  point  at  which  enough  individuals 
have  adopted  an  innovation  so  that  the  innovation’s  further  rale  of  adoption  becomes  self-sustaining.” 
Unfortunately,  little  contemporary  critical  mass  data  exists  to  aid  administrators  and  practitioners  in  planning  for 
increased  implementation  and  utilization. 

Problem/Significance 

During  the  summer  of  1996,  a research  study  of  465  Schools,  Colleges,  and  Departments  of  Education 
(SCDEs)  was  conducted  to  determine  whether  critical  mass  for  the  use  of  e-mail  and  Web  technologies  had  been 
reached  by  SCDE  faculty,  SCDE  students,  and  SCDE  institutions  (combining  data  from  both  independent  user 
groups).  Markus  (1990,  1987)  describes  interactive  technologies  as  any  which  require  reciprocal  communication; 
where  the  spread  of  their  use  relics  on  the  users’  contributions  of  information.  This  study  concentrated  specifically 
on  electronic  mail  and  Web  technologies  (Bulletin  Board  Systems  (BBS),  computer  conferencing,  the  Internet)  as 
the  interactive  media.  The  lack  of  documentation  regarding  current  use  of  information  technologies  and  critical  mass 
levels  leaves  educational  institutions  without  important  information  necessary  to  weigh  the  advantages  and 
disadvantages  of  continued  investments  in  technologies  infrastructure.  The  availability  of  information  technologies 
and  the  documented  increase  in  the  number  of  users  of  information  technologies  across  general  populations  indicates 
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that  core  requirements  arc  in  placr-  to  achieve  critical  mass  (Geoghegan,  1994;  Green  1 996b).  This  study 
investigated  critical  mass  for  information  technologies  specifically  within  SCDEs. 

Literature  Base 

Critical  mass  is  a conceptual  construct  of  diffusion  theory  which  has  tremendous  influence  on  the  practical 
applications  of  systemic  change  models.  Many  researchers  and  change  theorists  (e.g.,  Rogers,  1983,  1995;  Markus, 
1990;  Van  dc  Ven  & Rogers,  1988;  Valentc,  1993,  1995;  Zaitman,  Duncan,  & Holbcck,  1973)  have  explored  critical 
mass  as  an  indicator  and  a predictor  of  successful  adoption  efforts  within  a wide  variety  of  disciplines  and  settings. 

There  are  increasing  pressures  on  teacher  preparation  institutions  to  prepare  new  teachers  to  effectively  use 
technology  (Ely,  1996).  The  Office  of  Technology  Assessment  (OTA,  1 995'  continues  to  call  for  reforms  which 
upgrade  the  quality  of  teacher  preparation,  with  emphasis  on  increasing  the  implementation  and  integration  of 
technology  within  classrooms.  One  result  has  been  that  colleges  and  universities  are  embracing  information 
technologies  to  address  the  public  pressure  for  reform  in  the  way  teachers  deliver  instruction  (DeLougliry,  1996). 
While  computers  and  information  technologies  are  commonly  used  as  research  tools  in  higher  education  settings, 
Denk,  Martin  and  Sarangarm  (1993)  have  shown  that  computers  and  information  technologies  arc  not  an  integral 
part  of  classroom  instruction. 

The  work  of  Kenneth  Green  (1997,  1996a,  1996b)  and  colleagues  (Green  & Gilbert,  1995;  Green  & 
Eastman,  1994)  has  documented  critical  mass  levels  for  institutions  of  higher  education  and  access  to  computer- 
based  technologies.  This  study  focused  on  documenting  critical  mass  for  access  and  use  of  interactive  information 
technologies  within  pre-service  teacher  preparation  institutions. 

Limitations 

The  limitations  of  this  study  were  the  sample  identification,  survey  instrument  and  distribution,  and 
knowledge  base  of  survey  respondents.  The  sample  was  the  membership  of  an  intact  national  organization 
consisting  of  private  and  public  four-year  schools,  colleges,  and  departments  of  education.  Using  this  sample  placed 
a limitation  cn  generalizing  to  SCDEs  which  arc  not  members  of  the  intact  organization.  The  potential  for  a low 
response  return  rate  with  a mailed  survey  jeopardizes  representativeness  and  generality  due  to  the  selective  nature  of 
the  non-respondents. 

The  survey  instrument  was  created  by  the  research  team.  Two  recognized  experts  in  the  field  of  change  in 
schools  examined  the  survey  for  face  and  content  validity.  The  SCDE  Technology  Survey  was  sent  to  member 
institutions  of  the  national  organization  as  part  of  a larger  survey  and  submission  was  completely  voluntary. 

The  survey  respondent’s  knowledge  base  was  a potential  limitation.  The  larger  surveys  were  completed  by 
an  administrator  within  the  member  SCDE.  One  limitation  is  that  the  administrator  may  not  have  known  cnougii 
about  information  technologies  to  respond  accurately.  A second  is  that  the  administrator  may  have  completed  the 
survey  based  upon  their  personal  beliefs  or  perceptions  of  the  SCDE's  use  of  information  technologies  and  not  upon 
actual  use  of  information  technologies  within  the  SCDE.  Inter-woven  in  this  limitation  is  that  the  administrator 
answered  on  behalf  of  SCDE  students  and  faculty  based  on  their  knowledge,  observation,  and  perceptions  of 
information  technologies  use. 

Methodology 

A well-respected  professional  .teacher  education  organization  commissioned  the  design  and  analysis  of  a 
survey  to  gather  data  about  technology  use  within  SCDEs.  The  one  page  (front  and  back)  survey  was  mailed  (by  the 
sponsoring  organization)  to  744  member  institutions.  These  researchers  received  465  (63%)  useable  responses  from 
the  United  States,  Guam,  and  Puerto  Rico.  Questionnaires  were  completed  by  an  administrator  in  the  SCDE.  Of  the 
23  questionnaire  items,  15  were  used  to  compile  the  data  for  determining  critical  mass  levels.  Details  of  the  coding 
are  available  if  requested  (Tharp,  1997).  Description  and  analyses  of  the  additional  data  collected  are  available  in 
report  form  (Persichitte,  Tharp,  & Caffarclla,  1 997).  The  four  research  questions  associated  with  critical  mass  were 
answered  by  calculating  an  observed  critical  mass  value  for  combined  SCDE  faculty  and  student  use  of  e-mail  and 
Web  technologies,  for  SCDE  faculty  use  of  e-mail  and  Web  technologies,  for  SCDE  student  use  of  e-mail  and  Web 
technologies,  and  for  the  existence  of  required  infrastructure  for  the  use  of  c-maii  and  Web  technologies. 

Users  and  nonusers  (faculty,  student,  combined  faculty/student)  and  adequate/inadequale  infrastructure 
were  determined  by  combining  data  from  five  questionnaire  items  for  each  of  the  data  groupings.  If  the  combined 
value  was  greater  than  or  equal  to  60%,  the  institution  was  coded  ns  a ‘‘user”  and/or  “adequate.”  The  60%  cut-off 
was  chosen  to  parallel  Green’s  (1996a)  research  of  technology  use  at  two-year  and  four-year  higher  education 
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institutions.  This  coding  provided  the  data  for  the  observed  critical  mass  levels.  These  observed  critical  mass  values 
were  compared  to  a pre-determined  critical  mass  level  of  16%  to  conclude  whether  critical  mass  had  been  reached. 
The  decision  to  compare  the  observed  critical  mass  value  with  a critical  mass  value  of  16%  was  made  based  on 
reviewing  previous  statistical  analyses  of  critical  mass  conducted  by  contemporary  researchers  (Markus,  1987; 
Valente,  1995;  Rogers,  1995). 

Results 

Results  confirm  critical  mass  has  been  reached  in  SCDE  use  of  information  technologies  and  for  the 
provision  of  adequate  infrastructure  for  the  use  of  information  technologies,  for  all  the  research  questions,  the 
-bserved  critical  mass  values  greatly  exceeded  the  statistical  critical  mass  comparison  level  of  16%. 

RQ1:  Has  critical  mass  for  combined  SCDE  faculty  and  student  use  of  e-mail  and  Web  technologies  been 

reached? 

The  observed  critical  mass  value  for  this  question  was  87%  (405/465).  Critical  mass  has  been  reached  for 
combined  SCDE  faculty  and  student  use  of  e-mail  and  Web  technologies. 

RQ2:  Has  critical  mass  for  SCDE  faculty  use  of  e-mail  and  Web  technologies  been  reached? 

The  observed  critical  mass  value  for  this  question  was  92%  (429/465).  Critical  mass  has  been  reached  for 
SCDE  faculty  use  of  e-mail  and  Web  technologies. 

RQ3:  Has  critical  mass  for  SCDE  student  use  of  e-mail  and  Web  technologies  been  reached? 

The  observed  critical  mass  value  for  this  question  was  90%  (420/465).  Critical  mass  has  been  reached  for 
SCDE  student  use  of  e-mail  and  Web  technologies. 

RQ4:  Has  critical  mass  for  the  required  infrastructure  for  the  use  of  e-mail  and  Web  technologies  in  SCDEs  been 

reached? 

The  observed  critical  mass  value  for  this  question  was  95%  (444/465).  Critical  mass  has  been  reached  for 
the  required  infrastructure  for  the  use  of  e-mail  and  Web  technologies  in  SCDEs. 

Conclusions 

The  results  of  this  study  document  that  critical  mass  has  been  reached  in  student  use,  faculty  use,  combined 
faculty  and  student  use,  and  the  provision  of  adequate  infrastructure  for  e-mail  and  Web  technologies  across  SCDEs. 
Results  indicate  that  the  SCDE  adoption  rate  (92%)  has  surpassed  the  overall  higher  education  adoption  rate  (59%) 
cited  in  Green’s  research  (1996a,  1997).  MacKnight  (1995)  argues  that  once  SCDEs  have  reached  critical  mass  for 
information  technologies  infrastructure,  there  is  an  opportunity  to  progress  from  being  hardware  access  centers  to 
becoming  information  centers.  'The  issue  which  now  faces  SCDE  administrators  is  no  longer  whether  faculty  and 
students  will  use  the  technology,  but  how  they  will  use  the  technology  in  the  classroom  (Awbrey,  1996;  Cummings, 
1996). 

Recommendations 

For  those  SCDEs  which  have  adequate  infrastructure  in  place  (or  are  approaching  that  goal):  create  an 
information  technologies  plan  which  focuses  on  classroom  implementation  and  integration.  This  study  confirms  that 
the  number  of  adopters  of  information  technologies  is  high  enough  to  ensure  continued  adoption  of  the  technologies. 
Continued  use,  however,  should  be  supported  by  a three-year  life-c^  dc  technology  plan,  as  opposed  to  the  standard 
five-year  strategic  planning  model,  to  provide  for  the  update  and  replacement  of  obsolete  hardware,  software,  and 
networking.  Another  factor  likely  to  influence  continued  use  by  existing  adopters  is  the  availability  of  faculty 
training  for  the  use  and  integration  of  these  information  technologies  which  continue  to  change  so  rapidly.  The 
importance  of  continual  opportunities  for  faculty  at  all  levels  and  in  all  disciplines  to  engage  in  updating  their 
personal  and  pedagogical  skills  related  to  technology  use  cannot  be  over-emphasized.  Finally,  for  institutions  of 
higher  education,  continued  utilization  and  implementation  of  information  technologies  by  faculty  may  hinge  on 
revised  incentive  plans  which  recognize  and/or  reward  faculty  for  the  significant  knowledge,  time,  and  initiative 
required  to  reach  high  levels  of  technology  integration  within  instruction. 


Special  Thanks 

This  research  effort  was  sponsored  by  the  Research  and  Information  Committee  of  the  American 
Association  of  Colleges  for  Teacher  Education  (AACTE). 


References 

Awbrey,  C.  R.  (1996,  Winter).  Successfully  integrating  new  technologies  into  the  higher  education 
curriculum.  Educational  Technology  Review,  5.  7-17. 

Cummings,  L.  E.  (1996,  Winter).  Educational  technology-A  faculty  resistance  view:  Part  II:  Challenges  of 
resources,  technology,  and  tradition.  Educational  Technology  Review.  5.  18-20. 

DeLoughry,  T.  J.  (1996).  Reaching  a critical  mass.  The  Chronicle  of  Higher  Education.  47(20).  A17-A2Q. 

Dcnk,  J.,  Martin,  J.,  & Sarangarm,  S.  (1993).  Not  yet  comfortable  in  the  classroom:  A study  of  academic 
computing  at  three  land-grant  universities.  Journal  of  Educational  Technology  Systems.  22(1).  39-55. 

Ely,  D.  P.  (1996).  Trends  in  educational  technology.  Syracuse,  NY:  ERIC  Clearinghouse  on  Information  & 
Technology. 

Geoghegan,  W.  H.  (1994).  What  ever  happened  to  instructional  technology?  Norwalk.  CT:  International 
Business  Schools  Computing  Association. 

Green,  K.  C.  (1997).  Campus  computing  1996.  Encino,  CA:  Campus  Computing. 

Green,  K.  C.  (1996a).  Campus  computing  1995.  Encino,  CA:  Campus  Computing. 

Green,  K,  C.  (1996b).  The  coming  ubiquity  of  information  technology.  Change.  28(2).  24-3 1 . 

Green,  K.  C.,  & Gilbert,  S.  W.  (1995).  Content,  communications,  productivity  and  the  role  of  information 
technology  in  higher  education.  Change,  27(2).  8-18. 

Green,  K.  C.,  & Eastman,  S.  (1994).  Campus  computing  1993:  The  USC  national  survey  of  desktop 
computing  in  higher  education.  Los  Angeles:  University  of  Southern  California. 

MacKnight,  C.  B.  (1995).  Managing  technological  change  in  academe.  Cause/Effect.  8(1).  29-39. 

Markus,  M.  L.  (1990).  Toward  a “Critical  Mass”  theory  of  interactive  media.  In  J.  Fulk  & C.  W.  Steinfield 
(Eds.).  Organizations  and  communication  theory  (pp.  194-218).  Newbury  Park,  CA:  Sage  Publications. 

Markus,  M.  L.  (1987).  Toward  a “Critical  Mass”  theory  of  interactive  media.  Communication  Research. 
14(5),  491-511. 

Office  of  Technology  Assessment.  (1995).  Teachers  & technology:  Making  the  connection.  Washington, 
DC:  U.  S.  Government  Printing  Office. 

PcrsichiUe,  K.  A,,  Tharp,  D.  D.,  & Caffarella,  E.  P.  (1997).  The  use  of  technology  hv  schools,  colleges,  and 
departments  of  education:  Fall,  1996.  Washington,  DC:  American  Association  of  Colleges  for  Teacher  Education. 

Rogers,  E.  M.  (1995).  Diffusion  of  innovations  (4th  ed.).  New  York:  Free  Press. 

Rogers,  E.  M.  (1983).  Diffusion  of  innovations  (3rd  ed.).  New  York:  Free  Press. 

Tharp,  D.  D.  (1997).  Documenting  critical  mass  for  the  use  of  interactive  information  technologies  in 
schools,  colleges,  and  departments  of  education.  Published  dissertation,  University  of  Northern  Colorado,  Greeley, 
CO. 

Valente,  T.  W.  (1995).  Network  models  of  the  diffusion  of  innovations.  Cresskill,  NJ:  Hampton  Fress. 

Valente,  T.  W.  (1993).  Diffusion  of  innovations  and  policy  decision-making.  Journal  of  Communication. 
43(1),  30-41. 

Van  De  Ven,  A.  H.,  & Rogers,  E.  M.  (1988).  Innovations  and  organizations.  Communications  Research. 
15(5),  632-651. 

Zaltman,  G.,  Duncan,  R.,  & Holbeck,  J.  ( 1 973).  Innovations  and  organizations.  New  York:  John  Wiley. 


318 


31G 


Conditions  that  Facilitate  Teachers'  Internet  Use  in  Schools  with  High 
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Abstract 

Examines  the  presence  of  Ely’s  ( 1976 , 1990)  conditions  that  facilitate  innovation,  as  reported  by  Internet - 
using  teachers  in  leading-edge  schools . Descriptive  data  from  a national  survey  of  teachers  (N=23S)  in 
approximately  half  of  the  250  schools  in  the  National  School  Network  (NSN)  are  reported,  along  with  analysis  of  the 
correlation  of  the  condition  measures  with  teacher  and  student  Internet  use . Connections  are  drawn  to  other  NSN- 
related  studies  and  plans  for  further  analysis  of  the  covariance  of  the  conditions  and  interaction  effects  are 
described. 

Background 

The  history  of  educational  technology  is  filled  with  promises  of  technological  innovation  being  offered  as  a 
way  to  improve  teaching  and  learning.  Today  we  are  in  the  midst  of  yet  another  high-stakes,  government  and 
industry-sponsored  effort  to  introduce  technology  to  bring  about  educational  change.  The  federal  government,  states, 
and  private  organizations  are  spending  unprecedented  amounts  of  money  for  the  advancement  of  Internet  use  in 
schools  — offering  the  hope  of  more  to  win,  and  the  risk  of  more  to  lose  for  those  who  are  concerned  with 
educational  change  and  technology. 

Many  claim  that  the  Internet  is  a revolutionary  technology  for  learners  because  of  its  ability  to  provide 
resources,  data  sharing,  and  communication  to  all  who  arc  connected  (Hunter,  1993,  1995a,  1997a).  It  is  instructive 
to  remember,  however,  that  previous  claims  for  the  superiority  of  new  technologies  for  education  have  often 
foundered  not  on  the  quality  of  the  technologies  themselves,  but  on  complex  issues  related  to  implementation  in 
schools  (Berman,  1981;  Fullan,  1993;  Holloway,  1996;  McLaughlin,  1990). 

The  informants  for  this  study  are  Internet-using  teachers  in  a select  group  of  schools  that  have  already 
achieved  a high  level  of  Internet  connectivity.^  These  teachers  are  perhaps  the  best  informants  for  what  conditions 
facilitate  Internet  use  by  students  and  teachers,  i.e.,  once  a school  has  already  been  connected  to  the  Internet.  As 
more  schools  obtain  Internet  connectivity  the  issues  addressed  in  this  study  may  become  increasingly  important. 

By  examining  teachers  who  are  striving  to  use  the  Internet,  this  paper  avoids  problems  associated  with 
those  who  have  not  adopted  (Rogers,  1983)  or  who  may  be  resistant  to  using  these  new  technologies.  Finally,  while 
implementation  is  often  discussed  at  an  organizational  level,  the  variables  for  study  arc  viewed  as  teacher-level 
variables  with  the  teacher  as  the  unit  of  analysis , partly  because  one  might  expect  different  experiences  and 
perceptions  reported  by  teachers  in  the  same  school  (Becker,  1994a).  Thus,  this  study  might  be  understood  as  an 
examination  of  individual  teacher  behavior  (extent  of  use)  and  conditions  that  may  be  determinant  for  those  who  are 
among  the  strongest  Internet  users. 

Framework  of  Analysis 

Research  demonstrates  that  many  issues  influence  the  implementation  of  educational  technology 
innovations.  "The  more  factors  supporting  implementation,  the  more  change  in  practice  will  be  accomplished" 
(Fullan,  1991,  p.  67).  This  study  utilizes  a reasonably  holistic  framework  developed  by  Ely  (1976,  1990),  one  of  the 
few  frameworks  available  in  the  literature  (Holloway,  1996).  Ely's  claim  is  that  lack  of  any  of  these  eight  conditions 
can  hinder  use  of  an  innovation: 


3 The  schools  are  part  of  the  National  School  Network  (NSN)  project  organized  by  researchers  at  BBN  in 
Cambridge,  MA  in  1994  with  funding  the  National  Science  Foundation  (Contract  ft  RED-9454769). 
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1)  dissatisfaction  with  the  status  quo 

2)  existence  of  knowledge  and  skills 

3)  availability  of  resources 

4)  availability  of  time 

5)  existence  of  rewards  or  incentives 

6)  expectation  and  encouragement  of  participation 

1)  commitment  by  those  who  are  involved,  and 

8)  evidence  of  leadership  (Ely,  1990) 

This  framework  appears  to  be  broadly  gcncralizable,  readily  applied  in  diverse  educational  settings,  from 
school  districts  (Read,  1 994),  to  libraries  (Ely,  1976)  to  universities  (Bauder,  1993),  in  the  United  States  and  abroad 
(Ely,  1990).  A scries  of  dissertation  studies  also  support  its  usefulness  for  a variety  of  educational  innovations 
(Bauder,  1993;  Jeffrey,  1993;  Read,  1994;  Riley,  1995;  Stein,  1996).  Perhaps  more  importantly,  the  utility  of  this 
framework  is  supported  by  the  literature  related  to  computer  use  in  schools,  i.e.,  many  of  the  studies  one  sees  include 
variables  that  seem  to  be  consistent  with  Ely's  conditions  ~ c.g.,  dissatisfaction  (Barker  and  Taylor,  1993); 
knowledge  and  skills  (Shcingold,  el  al.,  1981);  resources  (Becker,  1994b;  Office  of  Educational  Technology,  1996); 
the  availability  of  time  (Honey  and  Henriquez,  1993;  Sheingold  and  Hadley,  1990),  and  so  on.4 

Methods 

This  study  reports  data  from  a national  survey  of  teachers  in  approximately  250  schools  registered  in  the 
National  School  Network  (NSN),  a project  funded  by  the  National  Science  Foundation  since  1994.  The  schools 
were  required  to  have  had  at  least  10  simultaneous  LAN-based  Internet  connections  for  a year  prior  to  joining  the 
project  and  to  be  nominated  by  an  intermediary  organization  involved  in  Internet-based  school  reform  (Hunter, 
1995b).  ''Internet-using  teachers"  include  teachers,  media-specialists,  or  other  professional  staff  who  do  any  of  the 
following: 

• Have  students  in  their  class  use  the  Internet,  cither  in  their  classroom  or  elsewhere  at  school; 

• Supervise  students  of  other  teachers  in  Internet  use  and  arc  at  least  partly  responsible  for  the 
activities  the  students  engage  in  while  using  the  Internet;  and/or 

• Use  the  Internet  themselves,  either  at  school  or  at  home,  for  professional  purposes. 

School-level  contacts  were  mailed  a "Teachers  Sampling  Form"  requesting  the  names  of  up  to  10  of  the 
"strongest  Internet-using  teachers"  in  the  school.  Once  these  forms  were  returned  (response  rate  approx.  60%), 
three  teachers  in  each  school  were  sampled.  As  in  past  studies  by  Becker  (1994a),  stronger  users  were  over- 
selected,  with  probabilities  related  to  their  reported  extent  of  use  as  indicated  on  the  Sampling  Form.  The 
assumption  here  is  that  the  distribution  of  users  has  a "long  tail"  so  that  only  a few  users  are  very  strong  at  each 
school;  a random  selection  would  tend  to  miss  these  stronger  users.6  The  booklet  for  the  Internet-using  teachers  had 
13  pages  containing  54  questions,  and  approximately  60%  ^ere  returned  ~ after  data  cleaning  this  resulted  in  a total 
sample  size  of  238  teachers  from  1 24  different  schools  for  m s analysis. 


4 Other  topics  from  the  literature,  such  as  psychological  variables  (Marcinkiewicz,  1993;  Marcinkiewicz  and 
Regstad,  1996),  and  leadership  styles  (Hall  and  Hord,  1987)  are  not  included  in  the  current  analysis. 

5 Henry  "Hank"  Becker  at  University  of  California,  Irvine,  co-authored  and  supervised  the  study.  Network 
coordinators,  technical  coordinators,  administrators  and  teachers  not  listed  among  the  top  ten  Internet  users  were 
also  surveyed.  Copies  of  instruments,  descriptive  data,  and  reports  are  being  made  available  online: 
http://nsn.bbn.com/ 

nsnjearnings/survey.html. 

6 Along  these  lines,  data  cleaning  removed  approximately  30  teachers  who  failed  to  indicate  use  with  students  on  a 
series  of  screening  questions.  A sample  of ‘‘other”  teachers  suggests  that  approximately  one-third  of  teachers  not 
identified  as  among  the  top  lntertnet  users  in  their  school  would  have  met  the  minimum  requirements  for  inclusion 
in  this  study  had  they  completed  the  Internet-Using  teachers  survey. 
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Internet  Use 

This  study  explores  a wide  range  of  Internet  uses  by  teachers  and  students  (Eisenbcrg  and  Ely,  1993; 
Harris,  1994;  Honey  and  Henriquez,  1996).  This  includes  constructivist  practices  identified  in  leading-edge  schools, 
e.g.,  activities  highlighted  in  conferences  and  newsletters  shared  across  the  National  School  Network  project 
(Hunter,  1997b)  such  as  project-based  learning  activities  (email  projects,  tclementoring,  shared  investigations, 
students  publishing  on  the  Web,  collaborating  with  other  school  sites,  and  participating  in  live  events  over  the 
Internet). 

While  some  uses  of  an  innovation  may  be  more  sophisticated  (Hall  and  Loucks,  1977)  or  exemplary 
(Becker,  1994a)  than  others,  implementation  studies  using  Ely's  framework  have  typically  not  differentiated 
between  types  or  quality  of  use.  It  is  understood  that  teachers  will  use  the  Internet  differently.  This  study  employs 
frequency-based  measures  (e.g.,  number  of  hours,  how  often),  and  measures  of  the  breadth  of  use  (e.g.,  number  of 
students,  number  of  activities)  so  that  those  who  involve  more  students  and  and  who  participate  in  a wider  range  of 
activities  score  higher  on  use. 

To  identify  the  extent  of  Internet  use  by  teachers,  this  study  combines  information  from  seven  sets  of 
questions  in  the  teacher  survey  booklet  (comprising  approximately  25  different  response  items): 

• MAXUSE:  The  maximum  use  a teacher  made  of  the  Internet  in  his/her  classes,  on  a scale  from  l to  4, 

where  1 represented  no  use;  2,  voluntary  student  use;  3,  occasional  use  by  all  students;  and  4,  use 

by  all  students  on  at  least  five  occasions. 

• AVGUSE:  The  average  use  a teacher  made  of  the  Internet  across  all  his/her  classes. 

• REQDUSE:  The  frequency  with  which  the  teacher  requires  students  to  use  the  Internet. 

• NNETPROJ:  The  number  of  discrete  types  of  network  learning  activities  the  teacher  has  had  students 

participate  in  during  the  year  (from  a list  of  17  types  including  working  with  scientists,  tutoring 

students  by  e-mail,  doing  Web  searches,  etc.). 

• USE4PREP:  How  frequently  the  teacher  accesses  the  Internet  while  doing  class  preparation  work  during 

the  school  day. 

• SELFUSE:  How  frequently  the  teacher  engages  in  six  other  Internet-related  activities,  such  as  posting  a 

message  to  a newsgroup,  or  creating  or  editing  a World  Wide  Web  page  for  their  class  or  school. 

• FUNCTION:  How  many  of  five  functions  for  using  the  Internet  (e.g.,  professional  collcgiality — sharing 

new  ideas,  discussing  teaching)  occupies  the  teacher  for  at  least  an  hour  per  week. 

While  this  study  is  generally  more  concerned  with  an  overall  use  measure,  factor  analysis  seems  to  confirm 
that  two  related  factors  can  be  identified.  A "student  use"  factor  is  based  on  scores  on  the  First  three  items  listed 
above.  These  concern  the  extent  of  student  classroom  use.  The  last  three  items  listed  above  load  on  a different 
factor  called  “teacher  use*’.  Interestingly,  the  variety  of  activities  undertaken  with  students  (NNETPROJ)  loads  with 
the  "teacher  use"  items,  suggesting  that  the  variety  of  activities  undertaken  with  students  is  more  closely  related  to  a 
teacher’s  own  exploration  or  use.  As  a result,  the  “teacher  use”  factor  may  be  interpreted  more  as  reflecting 
exploratory  use  by  the  teacher.7 8 

Approximately  75%  of  the  teachers  reported  that  all  the  students  in  at  least  one  of  their  classes  had  used  the 
Internet,  with  approximately  20%  indicating  that  they  require  student  use  of  the  Internet  on  a weekly  basis.  Of  the 
variety  of  networking  activities  listed,  by  far  the  most  frequently  reported  uses  with  students  were  looking  at  World 
Wide  Web  sites  and  searching  for  information  online  (both  reported  by  over  90%  of  teachers).  The  next  most 
common  activities  for  students,  creating  Web  sites  or  participating  in  email  exchanges,  were  only  reported  by  about 
30%  of  teachers. 


7 Efforts  to  obtain  a framework  for  determining  “level  of  use”  (Hall  and  Loucks,  1977)  for  Internet  implementation 
were  not  successful  prior  to  development  of  this  study.  Partly  because  the  Internet  is  a relatively  new  innovation, 
any  use  by  teachers  is  considered  noteworthy.  Becker  (1997)  begins  to  examine  the  impact  of  Internet  use  on 
specific  teaching  practices,  such  as  having  students  work  on  longer  projects. 

3 The  overall  use  measure  includes  all  the  items  (standardized  reliability  alpha  - .81).  It  is  calculated  from  the  sum 
of  the  two  orthogonal  factors  ("student"  and  "teacher"  use)  which  is  equivalent  (r=1.00)  to  a single  factor  solution. 
Oblique  factor  analysis  reveals  the  extent  to  which  these  factors  are,  in  fact,  related  to  each  other  (r=.53).  Unless 
indicated  otherwise,  relationships  to  “overall”  use  and  oblique  factor  scores  are  reported,  i.e.,  not  controlling  for  the 
other  factor.  If  only  overall  use  is  reported,  no  striking  differences  were  observed  between  student  and  teacher  use. 


321 


319 


‘wGnce/ning  Jhcir  own  use,  most  teachers  (approx.  70%)  reported  spending  at  least  an  hour  per  week 
developing  *!  ar  Internet  skills  or  searching  the  Web  for  instructional  materials.  Only  one-third  reported  spending 
more  ;h«:i  one  hour  per  week  using  the  Internet  for  professional  collcgiality.  About  half  reported  having  ever  posted 
to  newsgroups/listservs  or  having  ever  created  a Web  page.  Fewer  than  20%  reported  that  they  had  ever  participated 
in  real-time  events  via  the  Internet,  such  as  text-based  chats  or  videoconferencing.  Additional  descriptive  analysis 
of  use  by  students  and  teachers  within  NSN  schools  is  provided  by  Becker  (1997)  and  Hunter  (1998). 

Operationalizing  Conditions 

The  conditions  are  understood  to  be  "global"  constructs  that  are  made  up  of  a variety  of  components.  The 
approach  has  been  to  try  to  "write  or  select  items  presumed  to  be  tapping  each  of  the  facets"  (Pedhazur  and 
Schmelkin,  p.  68).  Given  the  scope  of  the  project,  no  exhaustive  measurement  of  each  condition  is  possible;  instead 
a few  indicators  are  intended  to  suggest  the  extent  to  which  each  condition  might  be  present.  In  a few  cases 
exploratory  factor  analysis  seems  to  confirm  that  different  elements  of  a condition  are  being  measured. 

Organization  of  the  variables  by  condition  is  based  on  an  interpretation  of  Ely's  framework.  For  example, 
items  related  to  peer  use  (Becker,  1994a)  are  interpreted  as  belonging  to  the  commitment  condition,  an  indication  of 
support  for  Internet  use  by  others,  while  administrator  use  is  viewed  as  an  indicator  of  leadership  9 While  formative 
use  of  the  instrument  generally  focused  on  whether  key  issues  for  teachers  were  being  addressed  by  the  conditions 
items,  perhaps  more  attention  might  have  been  paid  to  the  subjects’  view  of  the  framework  itself.10 

Findings 

Findings  describe  the  extent  to  which  the  conditions  seem  to  be  present,  and  the  strength  of  relationship 
between  the  conditions  and  Internet  use  measures.  Correlations  are  discussed  in  terms  of  the  ability  to  predict  the 
extent  of  use  from  knowledge  of  the  conditions  measures;  this  suggests  that  a relationship  exists,  but  not  necessarily 
a causal  one  (the  findings  could  just  have  easily  been  discussed  in  terms  of  predicting  the  presence  of  conditions 
from  scores  on  use).  These  are  preliminary  findings  because  no  effort  has  yet  been  made  to  control  for  the  presence 
of  other  conditions,  to  explore  the  covariance  (multi-colinearity)  of  the  conditions,  or  to  examine  interactions  that 
might  be  more  predictive  of  use.  Addtional  analysis  might  also  seek  to  control  for  intervening  variables,  such  as 
grades  and  subjects  taught,  class  ability  level,  or  school-wide  demographics.11 

Condition  #1:  Dissatisfaction  with  the  status  quo 

Dissatisfaction  indicators  include  the  perception  that  all  students  would  benefit  from  Internet  use,  both  in 
general  and  in  terms  of  learning  outcomes.  In  addition,  dissatisfaction  with  the  status  quo  (in  this  case,  non-Internet 
use)  is  indicated  by  the  extent  to  which  the  teacher  affirms  a number  of  reasons  for  Internet  use,  choosing  from  a list 
of  13  options  (Table  1). 

Respondents  overwhelmingly  agreed  that  all  students  would  benefit  from  knowing  how  to  use  the  Internet, 
with  80%  indicating  that  they  strongly  agree.  Concerning  whether  students  would  enjoy  learning  more  leading  to 
greater  effort  and  accomplishment,  the  response  was  more  mixed,  although  still  generally  positive  (Figure  1). 
Responses  on  the  two  items  were  correlated  (r=.28),  and  the  mean  response  was  significantly  correlated  with  overall 
use  (r=.37).12 


9 The  next  stage  of  analysis  may  examine  the  empirical  validity  of  distinctions  such  as  these. 

10  Formative  use  included  completion  of  the  survey  by  a dozen  (12)  teachers  in  seven  (7)  Syracuse-area  schools 
accompanied  by  open-ended  interviews,  site  visits,  and  an  unpublished  district-level  case  study. 

11  Becker  and  Ravitz  (1997)  found  that  higher  ability  classes  may  be  favored  with  Internet  use,  particularly  in 
schools  with  larger  traditionally  disadvantaged  populations,  and  that  subject  and  grade-level  differences  may  exist  as 
well. 

12  Unless  indicated  otherwise,  all  correlations  reported  in  this  paper  are  p < .001,  two-tailed. 
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Figure  1.  Perception  that  students  would  benefit  from  Internet  use 


Ail  students  would  benefit 


Students  enjoy  learning  more  leading 


to  greater  effort  & accomplishment  (q3c) 
iooi 1 


1.0  2.0  3,0  4.0  5.0  6.0 


<*■ — strongly  disagree strongly  agree  - — > 


< strongly  dicagrea  strongly  agree  > 


The  difference  reflected  in  the  above  responses  may  reflect  alternative  rationales  for  use  (Hawkridge,  et  ah, 
1990)  such  that  benefits  other  than  pedagogical  ones  are  perceived.  This  interpretation  is  supported  by  analysis  of 
the  reasons  given  for  use.  The  most  prevalent  reason  among  the  teachers  seemed  to  concern  the  pervasiveness  of 
technology  in  society,  a rationale  that  may  not  necessarily  require  learning  outcomes.  u 

The  distribution  of  mean  responses  for  the  reasons  items  (reliability  alpha  =.83)  approximated  a normal 
curve  and  had  about  the  same  correlation  with  overall  use  (r=.35)  as  the  mean  “benefits"  score.  In  the  end,  an 
average  z-score  on  both  sets  of  items  (reliability  alpha  = .84)  was  an  even  stronger  predictor  (r=.44)  of  overall  use. 

Table  I . Reasons  for  Internet  use  ordered  by  mean  response14 


Q-31  Which  of  these  are  reasons  for  your  own  use  of  the  Internet  at  school?  How 
important  a is  it  for  you? 

Mean 

Med 

Md 

S.d. 

c)  to  prepare  students  for  life  in  an  increasingly  technological  society 

2.69 

3 

3 

.57 

g)  to  keep  up  with  new  technologies  yourself  to  gain  access 

2.47 

3 

3 

.72 

1)  to  resources  or  materials  that  are  not  available  in  textbooks  or  in  the  library 

2.43 

3 

3 

.76 

a)  to  increase  student  motivation  and  participation  in  their  own  learning 

2.39 

3 

3 

.82 

b)  to  provide  opportunities  for  students  who  do  not  have  computers  at  home 

2.37 

3 

3 

.86 

d)  to  give  students  the  skills  they  will  need  in  college 

2.22 

2 

3 

.87 

0 to  help  students  feel  more  a part  of  the  global  community 

2.22 

2 

3 

.93 

h)  to  find  out  about  new  teaching  practices  that  you  may  want  to  use  or  adapt 

1.93 

2 

2 

.90 

j)  to  reduce  your  professional  isolation  through  e-mail  or  collaboration  with  others 

1.49 

2 

2 

1.10 

m)  to  support  larger  school  change  efforts  by  using  the  Internet  as  a catalyst 

1.49 

n 

2 

1.10 

c)  to  fulfill  students’  and  parents'  expectations 

1.43 

i 

1 

1.02 

*i)  to  overcome  remoteness  or  geographic  isolation  in  your  school  or  community 

1.27 

i 

a 

1.19 

*k)  to  overcome  a lack  of  specialized  staff  or  limited  program  offerings  at  your  school 

.89 

i 

0 

1.01 

Key:  "Not  a ReasonVNA"  = 0;  "Very  important  reason"  = 3 

*Note.  Rural  and  poor  schools  are  somewhat  underrepresented  in  NSN  (Becker  and  Ravitz,  1997). 


13  Hawkridge’s  fourth  ratonale  - developing  workplace-related  skills  — was  not  asked  here,  however  Goldman  and  Laserna 
(1996)  found  this  to  be  an  important  rationale  within  the  NSN  schools  they  studied.  Exploratory  factor  analysis  suggests  that 
reasons  related  to  the  pervasiveness  of  technology  in  society  (items  c,  d,  e,  f,  g)  were  related  to  both  student  and  teacher  use. 
However,  reasons  related  to  use  of  Internet  as  a catalyst  for  school  improvement  (items  h,  i,  j,  k,  m)  only  correlated  with 
teacher  use  (r=.35),  while  reasons  related  to  other  student  benefits,  only  loosely  interpreted  as  a pedagogical  rationale  (items 
a,  b,  i)  were  associated  more  with  student  use  (r=.25). 

14  Data  in  all  tables  reflect  four  (4)  or  fewer  missing  cases  (98%  completion)  unless  indicated  otherwise.  Measures  of  central 
tendency  and  standard  deviations  are  based  on  the  scoring  key  shown  within  or  below  each  table. 
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Condition  #2:  Knowledge  and  skills 

Knowledge  and  skills  indicators  include  self-reported  skill  levels  with  respect  to  13  different  Internet 
activities.  In  addition,  teachers  were  asked  to  what  extent  they  were  sufficiently  prepared  to  use  the  Internet  with 
respect  to  five  skills  that  may  he  required  for  classroom  use.  The  most  prevalent  of  the  “Internet  skills’’  included 
using  a search  engine  and  sending  email  (Tabic  2);  the  most  prevalent  of  the  “classroom  skills”  involved  finding 
relevant  online  information  and  awareness  of  what  the  Internet  can  do  (Table  3). 

The  mean  for  all  items  (reliability  alpha=.93)  was  the  best  predictor  of  “teacher  use”  (r=.70)  and  was 
strongly  correlated  with  overall  use  (r=.54).  However,  factor  analysis  created  a single  factor  that  combined  the 
"classroom  skills"  items  with  two  from  the  “Internet  skills”  list  — searching  the  Internet  and  downloading  materials. 
This  factor  might  still  be  interpreted  as  “classroom  skills”  because  the  additional  items  represent  among  the  most 
frequent  kinds  of  use  with  students  (Becker,  1997).  This  factor  alone  was  the  best  predictor  of  student  use  (r=.32), 
an  equally  strong  predictor  of  overall  use,  and  a reasonably  strong  predictor  of  teacher  use  (r=.48).  The  two  other 
factors  — one  mvolving  Web  authoring  and  IRC/MOO,  and  the  second  involving  "other”  skills  (email,  ftp)  — might 
be  viewed  as  more  “technical”  in  nature;  these  correlated  with  teacher  use,  but  were  not  predictive  of  use  with 
students. 

Table  2.  Self-reported  ‘ Internet  skills”  for  13  activities  ordered  by  mean  response 


Q-40  How  would  you  rate  your  ability  to  do  each  of  the  following  things  related 
to  the  Internet? 

Mean 

Med  Md  S.d. 

c)  Use  a search-engine  like  Alta  Vista  to  find  information  you  need 

2.52 

3 3 

.86 

b)  Send  e-mail  to  groups  of  people  without  naming  each  person  each  time 

2.25 

3 3 

1.01 

a)  Send  attached  files  with  an  e-mail  message 

2.11 

2 3 

1.05 

i)  Download  and  read  file  saved  in  a specific  format  (e.g.,  Word,  Adobe  Acrobat) 

1.79 

2 3 

1.17 

e)  Subscribe  to  a Listserv  and  participate  in  discussions  with  others 

1.47 

1 3 

1.19 

d)  Find  a Usenet  newsgroup  and  discuss  topics  of  interest  on  it 

1.40 

1 

1.08 

0 Put  files  on  a server  for  others  to  access 

1.32 

1 0 1.19 

j)  Produce  a simple  Web  page  (e.g.,  containing  text  and  graphics) 

1.30 

1 0 1.24 

h)  Translate  graphics  into  a format  for  placing  on  the  Web 

1.10 

1 0 1.21 

k)  Produce  a complex  Web  page  (e.g.,  tables,  frames,  sounds,  animation) 

.76 

0 0 .98 

1)  Be  a Web  Master  for  a school  or  district  (c.g.,  design  and  manage  Web  sites) 

.61 

0 0 .95 

g)  Participate  in  discussions  on  IRC,  MOO,  or  MUD 

.45 

0 0 .79 

m)  Do  programming  for  Web  pages  (c.g.,  CGI,  PERL,  JAVA,  Shockwave) 

.42 

0 0 .80 

Key:  "None"  = 0;  "Low"  = 1 ; "Medium"  - 2;  "High"  = 3;  "Don’t  Know"  = 0 

Table  3.  Self-reported  " classroom  skills ” related  to  Internet  use  ordered  by  mean 

Q-41  To  what  extent  are  you  sufficiently  prepared  to  use  the  Internet?  Do 

you  have  sufficient ... 

Mean 

Med 

Md 

S.d. 

b)  Ability  to  seek  and  find  relevant  online  information 

2.40 

3 

3 

.70 

a)  Awareness  of  what  the  Internet  can  do 

2.37 

2 

3 

.72 

d)  Knowledge  of  how  to  use  the  Internet  in  the  curriculum 

2.03 

2 

2 

.79 

c)  Classroom  management  skills  related  to  Internet  use 

2.00 

2 

2 

.86 

c)  Ability  to  manage  large  amounts  of  electronic  information 

1.93 

2 

2 

.92 

Key:  "Seriously  lacking"  = 0;  "Somewhat  lacking"  = 1 ; "Generally  present"  = 2;  "Very  present"  = 3 


Condition  #3:  Resources 

Two  sets  of  resource  indicators  include  support  resources  — technical,  training  and  curriculum  support  - 
and  hardware  resources  (c.g.,  computers  and  connectivity).  Of  the  support  resources,  teachers  generally  reported 
that  technical  support  and  training  opportunities  were  sufficiently  present  (Tabic  4).  However,  resources  for 
Internet  use  in  the  curriculum  and  help  for  integrating  online  activities  into  the  curriculum  were  most  often  reported 
as  being  insufficiently  present.  The  mean  score  on  these  four  items  (reliability  alpha  = .80),  was  correlated  with 
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overall  use  (r=.17,  pc.Ol),  but  the  mean  of  the  two  curriculum-related  items  (correlated,  r=.58)  was  a stronger 
predictor  (r=.22). 

Table  4.  Resource-related  conditions  reported  as  being  sufficiently  present  ordered  by  mean 


Q-35  To  what  extent  are  the  following  conditions  sufficiently  present  to 
support  Internet  use  by  you  and  your  students? 

Sufficiently  present... 

Mean 

Med 

Mode 

S.d. 

e)  Technical  support  is  available  when  you  need  it 

4.05 

4 

5 

1.56 

k)  Training  opportunities  are  offered  to  develop  your  Internet  skills 

3.84 

4 

5 

1.62 

0 Curriculum  resources  for  Internet  use  arc  available 

3.41 

3 

2 

1.49 

1)  Help  is  available  for  integrating  online  activities  into  the  curriculum  (c.g., 
workshops  or  meetings  to  help  teachers  plan  activities) 

3.10 

3 

2 

1.54 

Key:  “Strongly  disagree”  = 1;  “Strongly  agree0  = 6.  (9-12  missing  cases,  includes  “Don’t  fCnow'V’DK”  responses) 


Concerning  hardware  resources,  teachers  were  asked  whether  or  not  better  Internet  access  in  the  future 
would  increase  their  use  (Table  5).  A majority  indicated  that  classroom  Internet  access  and  simultaneous  access  for 
20-30  computers  would  increase  their  use  “a  lot”.  Surprisingly,  those  who  scored  higher  on  use  were  more  likely  to 
report  that  improved  access  would  help  “a  lot”  - this  suggests  that  less  frequent  users  may  not  perceive  access  as 
being  a critical  issue,  while  more  frequent  users  might  take  greater  advantage  of  improved  access  if  it  were 
provided.  As  a result,  an  indicator  score  from  these  items  (reliability  alpha  = .87)  was  not  at  all  predictive  of  use  (r= 
-.03). 

Table  5.  Access-related  resources  would  increase  use  ordered  by  percent  indicating  ”a  lot ” 


Q-l  Which  of  the  following  features  of  the  Internet  or  conditions  for 

accessing  it  would  make  you  likely  to  use  the  Internet  more? 

% 

"A  lot" 

% 

"A  little" 

% 

"Not  at 
aHY’NA” 

a)  Internet  access  in  your  own  classroom  rather  than  elsewhere  in  your  school 

61 

5 

34 

c)  Simultaneous  Internet  access  for  20-30  computers  in  a computer  lab 

57 

14 

29 

d)  Access  to  the  World  Wide  Web 

55 

6 

39 

b)  High-speed  Internet  connection  rather  than  modems 

52 

11 

37 

c)  Electronic  mail  to/from  anyone  in  the  world  with  an  Internet  e-mail  address 

44 

14 

42 

Next,  teachers  were  asked  to  indicate  the  number  of  simultaneous  Internet  connections  in  the  location 
where  students  most  frequently  use  the  Internet  — inside  the  classroom,  outside  the  classroom,  or  both.  60% 
indicated  that  the  classroom  is  a location  where  students  most  often  use  the  Internet,  and  80%  indicated  that  students 
most  often  use  the  Internet  in  another  location  --  with  45%  providing  data  on  both  locations.  The  highest  number  of 
simultaneous  connections  reported  in  the  classroom  was  32,  hut  a full  half  reported  only  one  Internet-connected 
computer  (Figure  2).  The  mean  response  for  the  number  of  simultaneous  connections  in  a location  outside  the 
classroom  was  21 , however  some  reported  up  to  100  (Figure  3);  these  latter  cases  indicated  that  they  were  reporting 
for  multiple  locations  (e.g.,  "in  other  classrooms"  or  “labs”). 
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Figure  2.  Number  of  simultaneous  Internet  connections  in  J Figure  3.  Number  of  simultaneous  Internet 

the  classroom  (if  classroom  is  a location  where  students  \ connections  in  other  locations  (if  students  most  often 

most  often  use  the  Internet)  [ use  the  Internet  outside  the  classroom) 


# Internet  connections  in  the  classroom 


Mean  = 4.4;  Med  = 1 ; S.d.=  6.2  (n=  1 42,  60%) 


# Internet  connections  in  other  locations 
701 


0 10  20  30  40  50  60  70  00  9C  100 


Mean  = 21 .6;  Med  = 20;  S.d  = 17.97  (n=l  90,  80%) 
Note:  midpoints  shown,  first  bar  represents  < 5 


The  number  of  simultaneous  Internet  connections  in  the  classroom  was  correlated  with  overall  Internet  use 
(r=.26),  predictive  of  use  by  teachers  and  students.  When  limiting  the  analysis  to  those  who  answered  50  or  fewer 
(n=178),  the  number  of  simultaneous  Internet  connections  outside  of  the  classroom  was  only  significantly  correlated 
with  student  Internet  use  (r=.28),  particularly  when  controlling  for  teacher  use  (r=.39).  Because  more  than  half  of 
the  teachers  did  not  answer  one  of  these  items,  a more  useful  indicator  is  the  maximum  z-score  on  the  two  items  (the 
two  z-scores  were  correlated,  r=.29)  — i.c.,  whether  or  not  a teacher  reports  a larger  number  of  connections  than 
other  teachers  in  either  location.  This  indicator  was  also  predictive  of  overall  use  (r=.22). 

Finally,  teachers  were  asked  about  the  amount  of  RAM  memory  on  the  computer  they  use  most  frequently 
at  school.  A little  over  10%  reported  having  32MB  or  more.  About  half  reported  having  16MB;  one-quarter 
reported  8MB,  and  fewer  than  15%  reported  any  less  than  that.  When  the  amount  of  RAM  was  converted  to  an 
interval  (as  opposed  to  ratio)  variable,  it  was  moderately  correlated  with  overall  use  (r=.19,  p <.10). 

Condition  #4:  Time 

When  asked  whether  there  is  sufficient  time  to  support  Internet  use  - in  the  teacher’s  work  schedule  and  in 
the  curriculum  — there  was  strong  disagreement  on  both  items  (Figure  4).  The  response  concerning  time  in  the 
curriculum  was  a better  predictor  of  Internet  use,  even  though  teachers  were  more  likely  to  disagree  that  there  was 
sufficient  time  in  the  schedule.  Time  in  the  schedule  was  correlated  with  overall  use  (r=.21),  but  not  as  strongly  as 
time  in  the  curriculum  (r=.35).  The  two  items  were  correlated  (r=.55),  and  the  mean  response  was  also  predictive  of 
overall  use  (r=.32). 
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Figure  4.  Insufficient  time  available  for  teacher  Internet  planning  and  use  in  the  curriculum 


Sufficient  time  in  your  schedule 


Sufficient  time  in  the  curriculum 


strongly  disagree  ---  strongly  egrne > 


<- — strongly  disagree  — - strongly  agree  — > 


Teachers  were  also  asked  to  report  the  actual  amount  of  in-school  preparation  time  they  are  given  each 
week  as  part  of  their  work  schedule.  Approximately  20%  reported  1 hour  or  less  per  week.  The  most  frequent 
response  was  5 hours,  with  an  average  of  about  4 hours.  This  included  a few  teachers  who  reported  more 
preparation  time,  i.e.,  a cluster  of  about  10%  who  reported  6 or  more  hours/wcek,  even  up  to  10  hours. 

The  amount  of  reported  preparation  time  was,  surprisingly,  not  correlated  with  the  above  items  related  to 
the  perception  of  time  availability,  nor  did  the  amount  of  actual  preparation  time  correlate  with  any  of  the  Internet 
use  measures.  A possible  explanation  is  that  teachers  do  not  necessarily  have  access  to  the  Internet  during  their  in- 
school preparation  time.  This  might  be  an  example  of  a resource  variable  “access"  interacting  with  a time  variable 
“preparation  time’*,  a relationship  that  might  be  explored  further.  Although  not  a time  variable  per  se,  those  who 
reported  using  the  Internet  more  frequently  during  their  class  preparation  lime  (USE4PREP)  also  scored 
significantly  higher  on  student  use  (r=.33)  even  when  teacher  use  was  held  constant  (r=,l  1 , pc.  10). 

Condition  #5:  Rewards  & Incentives 

Rewards  and  incentives  indicators  include  two  sets  of  items  — the  first  involves  the  perceived  level  of 
benefit  for  students  with  whom  the  teacher  has  used  the  Internet.  While  this  variable  may  be  more  a result  of  use, 
not  a cause,  students  benefiting  is  viewed  as  an  important  intrinsic  motivation  for  continued,  ongoing  use,  i.e.,  after 
some  initial  attempts  have  been  made.  The  intention  was  to  have  some  indication  of  a perceived  reward  concerning 
the  teacher’s  relationship  with  students  (Mitchell,  ct  al.,  1987).  This  item  is  different  from  the  more  hypothetical 
dissatisfaction  measure  concerning  whether  all  students  would  benefit. 

Teachers  reported  that  most  of  their  students  had  benefited  greatly  from  their  use  of  the  Internet  (Table  6). 
Approximately  15%  of  the  teachers  reported  that  all  of  the  students  with  whom  they  had  used  the  Internet  had 
benefited  greatly;  nearly  all  (92%)  reported  that  more  than  half  had  benefited  at  least  "somewhat”.  Finally,  negative 
experiences  were  rarely  reported,  most  (62%)  indciated  that  none  of  their  students  had  a negative  experience  overall. 
The  percent  of  students  reported  as  having  “greatly  benefited”  was  significantly  correlated  with  overall  use  (r=.38). 

Table  6.  Mean  percent  of  students  reported  as  having  benefited  at  each  levels 


Q-25  Of  your  students  who  have  used  the  Internet,  estimate  the  percent  of  them  in  each  category: 


Negative  experience  overall  Benefited  very  little  Somewhat  benefited  Greatly  benefited 

3.3  10.2  29.7  56.7 


The  primary  educational  value  that  was  reported  involved  access  to  a huge  variety  of  curriculum 
information.  Approximately  half  of  the  teachers  reported  that  students  were  applying  themselves  for  longer  periods 
of  time,  taking  more  responsibility  for  their  own  learning  and  showing  greater  interest  in  world  events  and  foreign 
cultures  as  a result  of  Internet  use.  About  half  also  reported  a more  equal  distribution  of  expertise  among  students 
and  that  average  students  were  communicating  at  ‘gifted’  levels.  Fewer  (less  than  1/3)  indicated  benefits  in  terms  of 
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students  having  a deeper  understanding  of  the  ideas  the  encounter*  and  having  experiences  with  or  increased  interest 
in  the  adult  world.15 

The  second  set  of  indicators  concern  the  availability  of  various  “extrinsic”  rewards  or  incentives  that  might 
be  provided  to  teachers  by  the  school  or  district  (Table  7).  Of  the  items  listed,  those  that  were  reported  as  being 
generally  available  involved  the  provision  of  computers,  equipment  or  modems  to  teachers  who  arc  interested  in  the 
Internet.  In  addition,  release  time,  reimbursement  for  inscrvicc  courses,  and  public  recognition  were  reported  in 
about  half  (40-50%)  of  the  eases.  For  the  remaining  items  a majority  indicated  that  the  reward  or  incentive  was  “not 
available’1. 

The  mean  response  for  each  teacher  on  all  these  “extrinsic”  items  (reliability  alpha  =.70)  was  correlated 
with  overall  use  (r=.22).  For  those  items  that  showed  a relationship  with  use  (a,  c,  d,  f,  g and  h),  the  mean  score  was 
only  slightly  more  predictive  of  use  (r=.25).,fi  There  seemed  to  be  no  relationship  between  the  percent  of  students 
having  greatly  benefited  and  the  availability  of  extrinsic  rewards  or  incentives. 

Table  7.  Extrinsic  rewards  or  incentives  "somewhat”  or  "generally0  available  ordered  by  sample  mean 


Q-50  To  what  extent  are  each  of  the  following  rewards  and  incentives  % 

available  at  your  school  to  encourage  teachers’  use  of  the  Internet?  availabl 

Mean  Me  M S.d  c 

d d “at  all” 


a)  Teachers  who  show  they  arc  interested  are  provided  with  computers/cquipmcnt  2.08  2 3 .82  80 

b)  Computers  or  modems  are  loaned  to  teachers  for  use  at  home  1 .79  2 1 .77  58 

e)  Cost  of  in-service/graduate  credits  for  Internet  workshops/courscs  is  1.69  1 1 .78  50 

reimbursed 

c)  Release  time  (substitutes)  arc  provided  for  Internet  development  activities  1.53  I I .67  44 

f)  Public  recognition  given  for  leadership,  helping  other  teachers/slaff  use  Internet  1.53  1 1 .69  42 

d)  Paid  time  provided  for  after  school  hours  spent  on  Internet-related  activities  1.33  1 1 .59  26 

h)  District/school  has  contests  rewarding  innovative  technology-using  teachers  1.19  1 1 .50  14 

g)  Career-ladder  opportunities  for  tclccommunications-using  teachers 1.17  1 1 .42  16 


Key:  "Not  available"  = 1 ; "Somewhat  available"  = 2;  "Generally  available"  = 3.  (13  missing  cases) 


Condition  #6:  Participation 

Participation  indicators  include  the  extent  of  a teacher’s  involvement  in  setting  the  course  for  Internet 
implementation  in  their  school  or  district  — contact  with  decision  makers  concerning  Internet-related  issues  and  the 
extent  of  involvement  in  various  other  activities  supporting  implementation.  Concerning  direct  involvement  in 
decision  making,  respondents  were  most  likely  to  report  having  a trusted  colleague  to  whom  they  can  voice  their 
concerns.  While  they  generally  reported  having  an  opportunity  to  voice  concerns,  fewer  reported  having  their  input 
directly  sought  by  decision  makers,  e.g.,  before  decisions  arc  made  (Table  8). 


15  Network  coordinator  data  actually  show  a decrease  in  some  of  these  benefits  being  reported  since  1995,  including 
students  having  greater  interest  in  world  events  and  foreign  cultures,  a deeper  understanding  of  ideas  they  encounter, 
and  greater  interest  in  the  adult  world  (Becker,  1997). 

16  Of  course,  selecting  only  those  items  that  “work”  as  predictors  of  use  may  capitalize  on  chance;  the  utility  of  these 
particular  items  as  opposed  to  others  would  have  to  be  confirmed  in  follow-up  studies. 
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Table  8.  Extent  to  which  teacher  input  is  sought  by  decision  makers  ordered  by  sample  mean 


Q-32  To  what  extent  is  your  input  sought  by  decision 

% 

% 

% 

% 

makers  in  your  school  or  district  concerning  Internet- 

neve 

sometime 

oft  c 

alway 

related  issues?  Mea  Me 

S.d.  r 

s 

n 

s 

n d 

(a 

D 

(=2) 

(=3) 

(=4) 

b)  There’s  a trusted  colleague  to  whom  I can  voice  concerns  3. 1 8 3 

.89  5 

16 

34 

45 

a)  I have  opportunity  to  voice  concerns  to  decision  makers  2,72  3 

.86  6 

37 

37 

20 

.85  11 

40 

37 

12 

.86  17 

47 

26 

10 

Note.  Percent  of  valid  responses  shown,  rounded.  Key  included  in  table. 

(9-12  missing) 

Of  the  various  other  types  of  activities  in  which  a teacher  might  participate,  those  reported  most  frequently 

included  selection  of  hardware  and  software,  providing  support  to  other  teachers, 

and  working  on  curriculum 

integration  (Table  9).  Regarding  other  activities,  half  or  more  reported  that  they  were  not  involved  "at  all". 

Table  9.  Teacher  involvement  in  various  other  activities  ordered  by  mean  response 

Q >36  To  what  extent  have  you  been  involved  in  any  of  the 

% Not 

% 

% 

% 

following  activities? 

at 

all 

Slightly 

Generali 

Very 

Mean 

Me  S.d. 

(= 

i) 

(=  2) 

y 

(=4) 

d 

(=  3) 

a)  Reviewing,  selecting,  purchasing  hardwarc/software  products  2.67 

3 

1.08 

18 

27 

26 

29 

b)  Providing  services  to  other  teachers,  training  /tech  support  2.65 

3 

1.16 

23 

22 

22 

33 

0 Developing  ways  of  integrating  Internet  into  the  curriculum  2.32 

2 

1.10 

31 

25 

25 

19 

e)  Serving  on  Internet-planning  committee.  School  or  district  2.06 

2 

1.20 

48 

19 

12 

2! 

c)  Developing  products  for  others,  e.g.,  software  or  guidelines  1 .97 

2 

1.07 

47 

22 

20 

12 

d)  Developing  school  or  district-wide  policies  for  Internet  use  1 .93 

1 

1.11 

50 

21 

15 

14 

h)  Meeting  with  parcnls/comm unity  members  re;  Internet  use  1 .52 

1 

.81 

65 

21 

11 

3 

g)  Attending  school  board  meetings  to  discuss  Internet  issues  1 .36 

1 

.73 

76 

15 

6 

3 

Note.  Percent  of  valid  responses  shown.  May  not  total  to  1 00%  due  to  rounding.  Key  included  in  table. 


In  addition,  teachers  were  asked  if  regular  meetings  to  discuss  Internet  issues  were  sufficiently  present. 
Most  (75%)  respondents  tended  to  disagree  that  Internet-related  meetings  were  sufficiently  present,  many  (36%) 
disagreed  strongly;  this  was  among  the  least  sufficiently  present  of  the  items  reported  by  Ravitz  (1997).  While  this 
may  indicate  a desire  for  more  involvement  in  decision-making,  the  wording  of  the  question  was  not  specific  enough 
to  determine  this,  i.e.,  responses  could  also  indicate  a desire  to  have  others  types  of  Internet-related  meetings,  e.g., 
with  peers  regarding  curriculum  uses. 

Overall,  teachers  who  use  the  Internet  more  were  more  likely  to  participate  in  decision  making  and 
planning  activities.  A combined  z-scorc  using  the  items  from  all  three  questions  (reliability  alpha  =.88)  correlated 
with  overall  use  (r=.35).  Dropping  the  last  item,  which  provided  less  value  in  terms  of  predicting  overall  use  (r=.17, 
p<.0l),  the  mean  score  on  the  two  sets  of  items  was  more  predictive  of  teacher  use  (r=.52)  and  overall  use  (r=.42), 
than  it  was  of  student  use  (r=.24);  in  general,  this  condition  seems  to  be  more  closely  associated  with  teacher  use. 

Condition  #7:  Commitment 

Indicators  of  commitment  include  the  extent  to  which  there  is  sufficient  organizational-level  commitment 
for  Internet  use  and  to  which  various  stakeholder  groups  have  been  supportive  of  Internet  use  over  the  past  two 
years.  Additional  indicators  include  the  number  of  other  teachers  in  the  school  with  whom  the  teacher  regularly 
converses  about  the  Internet,  and  the  number  of  others  who  arc  estimated  to  be  using  the  Internet.  Finally,  teachers 
were  asked  about  the  overall  supportiveness  of  people  in  different  stakeholder  groups. 

The  "organizational-level”  commitment  indicators  (Tabic  10)  were  reported  as  being  generally  present,  this 
included  the  existence  of  a long  range  plan,  a substantial  budget  being  in  place  or  assured,  and  Internet  use  as  a 
priority  in  school  improvement  plans.  Approximately  70%  tended  to  agree  that  each  was  sufficiently  present,  with 
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most  respondents  agreeing  strongly.  Responses  to  these  three  items  were  correlated  with  each  other  (r  >.56)  and  the 
mean  response  on  all  three  was  predictive  of  overall  use  (r=.23).  The  perception  that  Internet  use  is  a priority  was 
more  closely  associated  with  increased  use  (r=.24),  while  the  existence  of  a substantial  budget  was  a slightly  less 
powerful  predictor,  although  still  statistically  significant  (r=.16,  pc.b2). 


Table  10.  Commitment  items  concerning  planning  for  Internet  use  are  present 


Q-35  To  what  extent  are  the  following  conditions  sufficiently  present  to 
support  Internet  use  by  you  and  your  students? 

Mean 

Med 

Md 

S.d. 

a)  A long  range  plan  for  Internet  development  and  use  is  in  place 

4.26 

5 

6 

1.55 

b)  A substantial  budget  has  already  been  approved,  or  is  assured 

4.05 

4 

6 

1.60 

c)  Internet  use  is  a priority  in  existing  school  improvement  plans 

4.36 

5 

6 

1.49 

Key:  “Strongly  disagree”  = 1 ; “Strongly  agree”  = 6.  (9-12  missing  values) 

Concerning  the  supportiveness  of  various  stakeholder  groups  (Table  II),  responses  generally  indicated 
support  for  Internet  use  from  each  of  the  listed  groups.  Vendor  or  corporate  sponsorship  (item  h)  was  the  only  of 
these  items  that  had  a significant  correlation  with  overall  use  on  its  own  (r=.22,  p<.02).  The  mean  of  all  these  items 
(reliability  alpha  = .86)  was  also  predictive  of  overall  use  (r=.  17,  p<.02). 


TableIJ.  Level  of  reported  support  by , «</  f stakeholders 


Q-33  Over  the  past  two  years,  how  supportive  have  people  in  the 
following  positions  been  in  promoting  the  school's  Internet  use? 

Mca 

n 

Me 

d 

Md 

S.d. 

Missing 

b)  Teachers  (c.g.,  on  committees,  talking  to  others,  demonstrating  uses) 

3.2 

8 

3 

3 

.6 

9 

5 

c)  District-wide  committee  or  task-force  (c.g.,  district-level  planning) 

3 .2 
x 

3 

4 

. 8 
5 

20 

0 District  technical  support  (c.g.,  install/maintain  computers/  networks) 

3 .2 
1 

3 

.9 

4 

15 

e)  District  administration  (e.g.,  policies,  hiring  people,  seeking  grants) 

3.1 

o 

3 

3 

.8 

8 

25 

d)  School  board  or  decision  making  body  (e.g.,  budgets,  dev.  policies) 

2.9 

8 

3 

3 

.9 

0 

20 

a)  Parents  (c.g.,  asking  for  Internet,  bond  issues,  attending  meetings) 

2.9 

1 

3 

3 

.9 

2 

28 

i)  District  or  school  curriculum  supervisors  (e.g.,  ideas/maierials  for  use) 

2.7 

3 

3 

3 

1 . 
o 

23 

g)  Local  community  members  (c.g.,  voting  funds,  interest,  activities) 

2 . 5 
2 

3 

3 

.9 

o 

54 

h)  Vendors  or  corporate  sponsors  (c.g.,  hardware,  software,  telecom 

2.4 

2 

2 

.9 

51 

corps) 

5 

5 

j)  Local  businesses  (c.g.,  partnerships,  inslruclional/lcchnical  support, 

2 . 1 

2 

2 

.9 

65 

interest) 

8 

0 

Key:  “Unsupportive"  = 1 ; “Strongly  Supportive”  = 4.  "Don’t  Know"  responses  treated  as  missing. 


Concerning  the  number  of  other  teachers  in  the  school  with  whom  a teacher  has  discussed  the  Internet  in 
the  past  month,  most  (80%)  reported  speaking  with  10  or  fewer  teachers,  however  approximately  10%  reported 
speaking  with  over  25  in  the  last  month.  This  was  a relatively  strong  predictor  of  overall  use  (r=.37),  especially 
after  a square-root  transformation  created  a more  normal  distribution  and  one  positive  outlier  was  removed  (r~\46). 
Finally,  the  estimated  proportion  of  other  teachers  at  the  school  perceived  to  be  Internet  users  (Table  12)  was  also 
significantly  correlated  with  overall  use  (r=.27). 
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Table  12.  Estimated  proportion  of  teachers  and  administrators  who  use  the  Internet 


Q-42  Approximately  how  iu«ny  of  the  teachers  and  administrators  at  your  school  do  you  think  use  the 
Internet?  Cirleone.,. 


Teachers  (Q-42a) 

Administrators  (Q-42b) 

“A  few” 

24% 

35% 

“About  half’ 

39% 

17% 

“Most” 

19% 

17% 

“Nearly  all” 

14% 

21% 

“Don’t  Know” 

2% 

9% 

Mean=2.2;  Med=2,  S.d.=l.Q;  Missing=8 Mean=2.3;  Med=2;  S,d.=1.2;  Missing-23 

Key:  “A  few”  = 1;  “About  half’  = 2;  “Most”  = 3;  “Nearly  All”  = 4;  “Don’t  know”  = Missing. 


Condition  #8:  Leadership 

The  last  condition  involves  the  extent  to  which  administrators  and  school  principals  are  seen  as  advancing 
use  of  the  Internet.  This  includes  the  estimated  number  of  administrators  who  use  the  Internet  themselves,  the 
perception  that  administrative  support  is  sufficiently  present,  and  the  overall  stance  taken  by  administrators  with 
respect  to  Internet  use.  An  additional  form  of  leadership  that  was  explored  concerned  a teacher’s  awareness  of  other 
individuals  who  may  have  made  extraordinary  efforts  to  bring  about  school  Internet  use. 

The  estimated  proportion  of  administrators  who  are  Internet  users  (Table  1 l,  above)  was  moderately 
correlated  with  overall  use  (r=.  15,  p<.05).  Generally,  administrative  support  for  Internet  use  was  reported  as  being 
"sufficiently  present"  (Figure  6);  responses  on  this  item  correlated  with  overall  use  (r=.14,  p<05).  Concerning  the 
overall  leadership  stance,  approximately  90%  indicated  that  the  principal  or  key  administrators  had  at  least  been 
generally  supportive  (Figure  7);  however,  this  was  not  a useful  predictor.  The  mean  z-score  on  these  last  two  items 
was  predictive  of  teacher  use  (r=.  14,  p<.05),  but  not  student  use  - the  perception  of  administrative  support  seems  to 
be  more  closely  associated  with  teacher  use  than  student  use,  possibly  because  of  additional  conditions  required  to 
support  use  with  students.  Still,  the  mean  z-score  on  all  three  ilems,  including  the  proportion  of  administrators  using 
the  Internet  (reliability  alpha  =.68)  was  predictive  of  overall  use  (r=.  15,  p<.02). 


Figure  6.  Administrative  support  is  sufficiently  present 
to  support  Internet  use  ? 

Administrative  support  & initiative  present? 


Figure  7.  Leadership  stance  taken  by  principal  or  key 
administrator 

Leadership  by  principal  or  Xey  administrator 


STANCE  (q34) 

<0i 


1 - opposed  3 - supported  5 - initiated 

2 - ignored  4 - encouraged 


Mean  4.2;  Med=  5;  S.d.=  1.56  Missing=4 


Mcan=  3.8;  Mcd=4;  S.d.=.98  Missing=9 


Concerning  others  in  the  community  making  extraordinary  efforts  to  bring  about  Internet  use  (Table  13), 
the  most  frequently  mentioned  types  of  individuals  tended  to  be  those  who  work  in  the  -hool  (principal,  technology 
coordinator/media  specialist,  teachers)  or  a district-level  specialist.  None  of  the  oi  . types  of  individuals  were 
mentioned  in  more  than  10%  of  the  cases.  Incidentally,  of  those  who  wrote  in  "other"  responses  (n=  15),  three 
respondents  chose  to  highlight  extraordinary  efforts  made  by  the  school  board  or  a school  board  member. 
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Based  on  a comparison  of  means  (ANOVA),  those  teachers  who  reported  extraordinary  efforts  by  a 
university  liaison  were  significantly  more  likely  to  score  higher  on  use  measures.  7 Other  important  leadership 
roles,  by  this  criterion,  included  that  provided  by  a principal  or  school  administrator,  and  that  provided  by  vendors  of 
network  software,  educational  content  or  computers. 

Table  1 3.  Extraordinary • efforts  by  individuals  in  various  groups  and  relationship  to  Internet  use  ordered  by 
reported  frequency 


Q-51  Is  there  someone  who  has  made  extraordinary  efforts  to  bring  to  reality  a 
vision  of  tele-communications  use  in  your  school— someone  who  has  made  a 
substantial  contribution  that  would  not  have  been  made  by  other  individuals  in 
that  same  role?  Circle  as  many  as  apply,  but  be  "tough. " 

# 

% 

Mean 
diff.  in 
overall 
use 

b)  School  computer  or  technology  coordinator  or  media  specialist 

60 

25 

.02 

c)  A teacher 

54 

23 

.09 

d)  District-level  technology  specialist  or  other  district  administrator 

40 

17 

.15 

a)  Principal  or  other  school  administrator 

31 

13 

.37** 

c)  Specific  parent(s) 

8 

3 

.07 

h)  University  faculty  member  or  students  or  network-based  project  liaison 

8 

3 

.81*** 

i)  A provider  of  network  software,  educational  content  or  computer  retailer 

7 

3 

.47* 

0 Local  business  person,  corporate  sponsor  or  professional  person  in  your  community 

7 

3 

-.12 

g)  Government  official 

0 

0 

.00 

*12  <.20  **p  < .06  ***J2<.02 


Interestingly,  those  who  reported  extraordinary  efforts  by  a local  business  person  or  community  member 
scored  slightly  lower  on  overall  use,  however  this  was  not  a statistically  significant  finding.  The  total  number  of 
items  selected  was  moderately  predictive  of  use  (r=.12,  p<.05),  while  the  number  of  the  three  significant  items  that 
were  indicated  was  a somewhat  better  predictor  (r=.19,  p<.005).  Of  these  three  items,  60%  of  all  respondents 
(n=141 ) reported  none,  35%  (n=82)  indicated  at  least  one,  and  only  5%  (n=  1 2)  responded  that  individuals  in  two  of 
these  roles  had  made  extraordinary  efforts  — all  but  one  indicated  the  principal  or  school  administrator,  paired  seven 
times  with  a provider,  and  four  times  with  a university  liaison. 

The  number  of  people  reported  as  making  extraordinary  efforts  was  correlated  with  the  mean  /.-score  on  the 
other  leadership  items.  The  mean  z-scorc  on  all  four  leadership  indicators  (reliability  alpha=.63)  was  a better 
predictor  of  overall  use  (r=.2(),  p<.005)  than  any  single  leadership  indicator. 

Summary 

For  each  of  Ely’s  conditions  (1976,  1990)  several  survey  items  were  developed  based  on  a review  of  the 
literature.  Descriptive  data  reported  here  suggest  which  conditions  seem  to  be  most  present  or  lacking  as  reported 
by  Internet-using  teachers  in  leading-edge  schools.  For  example,  it  seems  that  technical  and  training  resources, 
administrative  support  and  organizational  commitment  arc  among  the  most  present;  conditions  that  seem  to  he 
lacking  include  time  for  planning  and  curriculum  use  and  the  availability  of  curriculum-related  help  and  resources. 

In  general,  indicators  for  each  condition  are  significantly  correlated  with  use  measures,  including  both 
teacher  and  student  use.  Within  each  condition  certain  indicators  arc  stronger  predictors  than  others.  Some  that 
appear  to  be  more  predictive  of  student  use  include  classroom-related  knowledge  and  skills,  and  curriculum 
resources;  those  that  seem  more  predictive  of  teacher  use  include  more  "technical"  Internet  knowledge  and  skills, 
the  extent  of  participation  decision  making  and  other  implementation-related  activities,  and  the  perception  of 
support  for  Internet  use  by  principals  and  administrators.  The  conditions  that  seem  to  be  most  predictive  of  use 
overall,  based  on  measures  developed  here,  include  knowledge  and  skills  and  dissatisfaction  with  the  status  quo.  In 
addition,  a few  individual  indicators  from  other  conditions  seem  to  be  strong  predictors  --  the  number  of  other 
teachers  in  the  moo!  with  whom  a teacher  discusses  the  Internet,  and  the  number  of  students  reported  as  having 
benefited  greatly. 
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Goldman  and  Laserna  (1996)  provide  case  studies  of  NSN  schools  involved  in  university-school  partnerships. 
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The  next  stage  of  analysis  will  focus  on  the  covariance  of  indicators  across  the  conditions,  possible 
interaction  effects,  and  exploration  of  new  factors  that  may  cut  across  the  framework  — such  as  all  the  items  related 
to  curriculum  integration.  A regression  model  will  determine  the  extent  to  which  the  conditions  as  a whole  can 
account  for  differences  in  the  extent  of  teacher  Internet  use.  In  conclusion,  the  framework  may  help  shed  light  on 
the  complex  interplay  of  conditions,  all  of  which  seem  to  be  related  to  teacher  implementation  of  the  Internet,  the 
latest  in  a long  line  of  educalional  technology  innovations. 
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Abstract 

The  promise  for  the  successful  use  of  distance  technology  is  great.  However , distance  education  is  laden 
with  the  potential  for  student  anxiety,  brought  on  by  the  technology,  that  could  hinder  the  effectiveness  of  instruction 
in  such  an  environment.  This  study  seeks  to  determine  the  relationship  between  students'  perceptions  of  the  two- 
way  audio/video  classroom  and  their  anxiety'.  Additionally , it  seeks  to  determine  the  relationship  between  students' 
perceptions  of  the  two-way  audio/video  classroom  and  their  satisfaction  with  their  distance  learning  experience. 

222  students  in  two-way  audio/video  distance  classes  completed  inventories  that  measured  their : anxiety  in 
the  distance  situation,  satisfaction  with  learning  in  the  distance  learning  environment,  and  perception  of  key 
elements  in  the  two-way  audio/video  learning  environment.  The  data  collected  for  each  measure  (anxiety, 
satisfaction,  and  key  elements  in  the  environment)  was  then  correlated  and  step-wise  regression  analyses  were  run. 
The  findings  indicate  that  there  is  a positive  relationship  between  student  anxiety  in  the  distance  situation,  their 
satisfaction  with  learning  in  the  distance  learning  environment  and  their  perception  of  key  elements  in  the  two-way 
audio/video  learning  environment.  In  addition,  key  elements  of  the  environment  explain  a significant  portion  of  the 
variance  of  student  anxiety  in  the  two-way  audio/video  environment.  These  environmental  elements  also  explain  a 
significant  portion  of  the  variance  of  student  satisfaction  with  the  learning  experience. 

In  the  past  decade  we  have  seen  a technological  explosion.  This  in  turn  has  increased  the  growth  and 
development  of  distance  education  programs.  However,  an  unfortunate  side  effect  of  this  rapid  change  is  that  much 
of  the  population  has  been  left  behind.  As  a result,  people  from  all  walks  of  life  experience  techno,  or  computer, 
phobia  (Rosen  & Maguire,  1990;  Mahmood  & Medewitz,  1989).  Our  study  extends  the  ideas  of  earlier  authors  to 
an  environment  that  was  created  as  a result  of  this  technological  explosion:  two-way  audio/vidco  distance  education 
classes.  In  particular,  this  study  explores  the  environment  in  a two-way  audio/video  classroom,  its  relationship  to 
students'  anxiety,  and  ultimately  how  it  might  affect  interactivity  and  satisfaction  with  the  learning  experience. 

Issues  regarding  interactivity,  anxiety  and  satisfaction  and  the  quality  of  learning  experiences  arc  not  new. 
Educators  have  long  realized  the  importance  of  interaction  among  students  and  instructors  in  facilitating  the  learning 
experience.  However,  in  a two-way  audio/vidco  distance  education  environment  these  issues  take  on  an  even  greater 
importance.  Researchers  have  repeatedly  stressed  the  importance  of  real-time  interaction  among  people  in  a distance 
education  environment  (Barker,  Frisbie,  & Patrick,  1989;  Wagner,  1994).  Furthermore,  distance  education 
researchers  have  stressed  the  importance  of  student-instructor  interaction  in  order  to  decrease  anxiety  and  increase 
motivation  (Mclsacc  & Gunawardena,  1996;  Ehrman,  1990).  In  an  extensive  review  of  the  literature,  Mclsaac  and 
Gunawardena  (1996)  noted  that  nearly  one-fourth  of  the  literature  was  on  this  topic.  Obviously,  interaction  is  an 
integral  part  of  a distance  education  class.  Numerous  methods  can  be  applied  to  foster  interaction,  but  one  area  that 
should  not  be  ignored  is  student  comfort.  Student  comfort,  or  the  absence  of  anxiety,  is  essential  in  promoting 
interaction  in  a classroom. 

One  cause  for  concern  regarding  student  anxiety  in  a distance  education  environment  is  that  the  physical 
distance  between  classrooms,  the  students,  and  the  teachers  can  strip  the  class  participants  of  the  physical  presence 
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and  comfort  that  can  be  found  in  a traditional  classroom  (Moore  & Kcarslcy,  1996;  Bruce  & Shade,  1995).  Two-way 
audio/video  was  originally  seen  as  a solution  for  problems  associated  with  the  dispersion  of  members  of  the 
classroom.  In  essence,  two-way  audio/video  was  intended  to  promote  the  presence  normally  found  in  a face-to-face 
situation.  In  turn,  this  technology  was  also  intended  to  foster  better  interaction,  thus  creating  a better  and  more 
satisfying  learning  experience  for  the  students.  To  a certain  extent,  two-way  audio/video  has  met  these  expectations 
(Mclsacc  & Gunawardcna,  1996). 

These  findings  are  supported  by  communications  research.  Researchers  in  this  area  have  found,  when 
examining  interactions  between  people  over  two-way  audio/video  versus  face-to-face  situations,  that  people  tend  to 
communicate  equally  well  in  either  situation.  With  respect  to  communication,  there  were  little  differences  between 
the  two  mediums  (Sellen,  1995;  Heeter,  1992).  Unfortunately,  once  technology  is  added  to  an  educational  situation 
things  tend  to  lose  their  simplicity.  Realizing  this,  Hillman,  Willis  and  Gunawardcna  (1994)  pointed  out  that 
although  the  two-way  audio/video  environment  has  the  potential  to  provide  real-time  interaction,  significant 
impediments  might  result  if  the  student  is  not  comfortable  with  the  technology  or  if  the  technology  is  poorly 
implemented.  We  believe  that  anxiety  induced  b the  physical  environment  in  a two-way  audio/video  classroom  will 
affect  students’  interaction  and  in  turn  their  satisfaction. 

The  idea  that  anxiety  has  hindered  learning  is  well  established  (Eysenck,  1979;  Darke,  1988).  It  has  also 
been  established  that  when  students  arc  anxious,  they  will  feel  less  comfortable  speaking  in  groups  (Weinberger  & 
Engelhart,  1976).  In  sum,  given  a large  classroom  with  a number  of  anxious  students,  it  is  likely  that  the  overall 
quality  of  interaction  and  success  of  a class  will  suffer.  There  are  numerous  ways  to  combat  this  anxiety,  but  one 
way  is  to  establish  a safe  and  comfortable  environment.  In  a traditional  classroom  where  there  are  few  and  familiar 
environmental  variables,  this  is  relatively  simple.  However,  in  a two-way  audio/video  classroom  there  are  numerous 
and  less  familiar  variables.  In  addition,  the  use  of  technology  itself  can  be  quite  anxiety-provoking  for  some 
individuals  (Rosen  & Maguire,  1990;  Mahmood  & Mcdewitz,  1989). 

The  physical  environment  is  one  of  the  foundations  on  which  a comfortable  anxiety-free  classroom  is  built. 
Careful  planning  of  a two-way  audio/video  classroom  can  ease  communication  and  interaction;  failure  to  do  so  can 
be  not  only  unhelpful,  but  detrimental  (Cape,  1996).  Although  a plethora  of  techniques  can  aid  in  reducing  and 
facilitating  communication  and  interaction  in  the  classroom,  using  any  of  these  methods  in  the  absence  of  a sound 
foundation  is  analogous  to  a well-conccived  building  placed  on  a fault  line.  In  the  end,  these  interventions  would 
collapse  into  the  base  from  which  they  were  built.  Careful  planning  and  judicious  support  in  a two-way  audio/video 
classroom  play  an  important  part  in  establishing  a solid  foundation. 

There  arc  several  factors  that  should  be  addressed  in  order  for  a two-way  audio/video  class  to  be  successful. 
Factors  such  as  lighting,  audio  quality,  table  and  monitor  arrangement  can  all  affect  a student's  anxiety  level.  In 
addition,  unintelligible  audio  or  video  portions  of  the  class  can  also  create  a whole  new  set  of  stressors  that  may 
hinder  learning.  Curiosity  may  induce  some  students  to  enroll  in  a video  conference  course,  but  others  arc  likely  to 
be  unnerved  by  seeing  themselves  on  the  video  monitor  (Bruce  & Shade,  1995).  The  technology  in  this 
environment  can  be  particularly  disconcerting  for  people.  For  this  reason,  the  availability  and  response  time  of 
technical  support  staff  in  this  type  of  environment  is  also  crucial.  If  a problem  with  the  technology  arises  and  it  is 
not  resolved  quickly,  the  anxiety  and  frustration  level  of  all  participants  is  likely  to  increase.  Essentially  it  is 
important  that  the  technology,  environment,  and  support  work  together  seamlessly,  lest  the  environment  become 
intrusive  and  anxiety-provoking  thus  compromising  the  level  of  interaction. 

Although  two-way  ~udio/vidco  classrooms  have  been  around  for  over  40  years,  there  has  been  little 
empirical  examination  of  physical  environmental  variables  and,  in  particular,  students'  perceptions  and  satisfaction 
with  them  (Bincr,  Dean,  & Mcllinger,  1994).  According  to  Mclsaac  and  Gunawardcna  (1996) "...  research  is  needed 
to  identify  how  technology  interacts  with  students  and  how  it  affects  teaching  and  learning."  Additionally,  research 
on  how  the  technology  affects  learners  as  well  as  the  environmental  conditions  necessary  for  its  implementation 
should  be  ongoing  (Mclsacc  & Gunawardcna,  1 996.)  This  is  especially  true  as  the  technology  changes. 

In  this  study  wc  seek  to  develop  a method  by  which  to  assess  physical  and  environmental  aspects  as  well  as 
students  anxiety  in  this  type  of  environment  and  their  overall  satisfaction  with  their  learning  experience.  Although 
many  elements  make  up  a solid  foundation  in  the  classroom  environment,  we  believe  that  environmental  variables 
arc  particularly  relevant  to  student  anxiety  level,  satisfaction,  and  ultimately  student  success  in  the  classroom. 
Furthermore,  it  is  essential  that  educators  and  programmers  take  these  elements  into  account  when  developing  and 
conducting  distance  education  classes.  The  success  level  of  the  students,  the  instructors,  and  ultimately  the  distance 
education  program  can  only  be  as  good  as  the  foundation  on  which  it  lies. 
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Method 

Participants 

The  participants  for  the  study  were  students  enrolled  in  12  separate  two-way  audio  video  classes  at  two 
major  midwestern  universities  and  two  midwestem  community  colleges.  The  sample  consisted  of  222  subjects  (146 
female,  54  male,  22  non-reported;  93%  Caucasian,  4%  African  American,  \%  Hispanic,  1%  Asian,  1%  other).  The 
age  range  for  the  sample  was  18-64,  with  a modal  age  of  19  and  a mean  age  of  31.  97%  of  the  participants  were 
native  English  speakers. 

Instruments 

The  instruments  used  in  this  study  included  the  following  measures:  anxiety  in  a two-way  audio/video 
classroom,  general  satisfaction  with  the  distance  learning  experience,  and  student  perception  of  environmental 
elements  of  the  two-way  audio/video  classroom.  Environmental  variables  were  subdivided  into  three  categories: 
perception  of  the  physical  environment,  perception  of  the  physical  layout,  and  perception  of  the  management  of  the 
distance  learning  environment. 

The  measures  in  this  study  were  developed  due  to  the  lack  of  empirically  validated  instruments  that  assess 
these  constructs.  Developing  these  measures  was  a multi-step  process.  The  initial  items  were  developed  through  a 
combination  of  techniques.  In  the  ease  of  the  anxiety  measure,  items  from  the  STAI:  The  State-Trait  Anxiety 
Inventory  (Spielberger,  Gorusch,  & Lushene,  1970)  were  examined  and  then  modified  to  address  specifically  a two- 
way  audio/video  classroom  situation.  Additionally,  the  items  in  this  inventory  were  designed  in  such  a way  that  a 
high  score  would  represent  low  anxiety.  This  was  done  so  that  there  would  be  uniformity  among  the  different 
measures.  Thus,  a high  score  on  any  of  the  measures  would  represent  a positive  attribute. 

An  approach  similar  to  the  development  of  the  anxiety  measures  was  used  for  developing  the  satisfaction 
with  learning  experience  measure.  Consumer  satisfaction  has  been  an  important  area  of  research  in  many  fields 
(Ware,  1978;  Lebow,  1982;  Locke,  1976).  In  the  past  decade  this  area  of  research  has  gained  interest  in  the  area  of 
education  (Chadwick  & Ward,  1987).  As  competition  for  students  grows,  particularly  in  distance  education 
environments,  schools  will  need  to  pay  close  attention  to  this  variable  lest  their  programs  become  victims  of  low 
retention  and  returns.  Although  the  fields  studied  vary  each  seems  to  tap  a certain  unidimensional  factor  in  its 
assessment  of  consumer  satisfaction.  Larsen,  Attkisson,  Hargreaves,  and  Nguyen  (1980)  in  their  development  of 
Services  Evaluation  Questionnaire,  found  that  three  items  defined  a unidimensional  measure  of  satisfaction. 

1.  To  what  extent  has  our  program  met  your  needs? 

2.  In  an  overall  general  sense,  how  satisfied  are  you  with  the  services  you  received? 

3.  If  you  were  to  seek  help  again,  would  you  come  back  to  our  program? 

These  three  items  were  used  as  a guideline  for  the  initial  development  of  items  to  assess  satisfaction  with  learning 
experience. 

Finally,  in  developing  a measure  to  assess  environmental  aspects  of  a two-way  audio/vidco  classroom,  we 
interviewed  ten  distance  education  educators  and  four  technicians  from  two  large  midwestern  universities. 
Specifically  we  asked  them  questions  regarding  their  own  experiences  and  their  students’  perceptions  of  the  distance 
education  environment.  This  information  was  then  summarized  and  categorized  into  different  environmental  issues 
for  two-way  audio/vidco  environments.  Individual  items  were  derived  from  these  categories. 

Once  the  base  set  of  items  for  each  of  the  inventories  was  created,  three  experts  in  the  field  of  distance 
education  were  given  the  items  and  were  asked  to  rate  how  accurately  the  items  measured  each  of  the  constructs. 
Additionally,  a rating  procedure  was  used  in  which  66  novices,  students  in  educational  psychology  courses,  were 
given  definitions  for  each  constnict  and  asked  to  identify  which  construct  each  of  the  items  seemed  to  measure. 

The  definitions  used  for  each  construct  arc  as  follows: 

Anxiety:  Fearful  concern  about  performing  and/or  learning  in  a two-way  audio/vidco  classroom. 
Apprehension  regarding  one's  capacity  to  cope  with  a two-way  audio/vidco  environment,  despite 
having  adequate  preparation. 

Satisfaction  with  the  learning  experience:  How  satisfied  one  was  with  their  learning  experiences  in  a 
two-way  audio/video  classroom. 

Physical  Environment:  Perceptions  of  various  physical  characteristics  of  the  two-way  audio/vidco 
environment,  such  as  audio,  video,  and  so  on. 


Physical  layout:  Perceptions  of  the  physical  layout  of  a two-way  audio/video  classroom. 

Management  of  the  two-way  audio/video  classroom:  Perceptions  of  how  well  the  instructor  and/or 
technology  support  team  was  able  to  manage  successfully  a distance  learning  environment. 
Management  includes  the  support  and  use  of  the  two-way  audio/video  technology,  as  well  as  regular 
classroom  management. 

Using  these  definitions,  students  were  asked  to  categorize  each  item.  Participants  were  allowed  to  choose 
one,  multiple,  or  none  of  the  constructs  when  making  this  decision.  Next,  concordance  rates  among  these  students 
were  calculated. 

The  items  eventually  chosen  for  each  of  the  scales  had  a wide  range  of  frequency  ratings  from  the  novices. 
When  examining  these  results,  we  looked  at  two  specific  areas.  First  we  examined  the  percentage  of  identifications 
for  the  selected  definition  alone  and  then  we  looked  at  the  number  of  identifications  for  something  other  than  the 
selected  definition.  For  the  anxiety  items  there  was  a 64%  - 91%  selection  of  the  anxiety  definition  alone  with  no 
other  choices  or  combinations  above  15%.  For  the  satisfaction  with  learning  experience  there  was  a 26%-79% 
selection  of  the  satisfaction  with  learning  experience  alone.  The  two  items  with  the  smaller  confirmation  rates  also 
had  confirmations  of  25%  and  17%  for  the  no  matches  choice.  The  environmental  variables,  almost  uniformly,  were 
confirmed  as  one  or  a combination  of  the  three  environmental  choices.  The  overall  percentages  for  confirmations  as 
one  or  a combination  of  the  three  environmental  variables  ranged  from  48%  - 98%>,  with  only  two  of  these  items 
below  the  75%  rate.  However,  when  these  items  were  broken  down  into  the  three  individual  environmental 
categories  the  novices  confirmation  rate  dropped  considerably.  The  only  category  with  reasonable  sole  confirmation 
rales  was  the  physical  layout,  with  confirmation  rates  ranging  from  60%-85%.  Of  these  items  only  one  item, 
environmental  perception,  had  an  additional  choice  equal  to  or  above  15%. 

The  concordance  rates  of  all  of  these  items  were  examined  in  conjunction  with  the  expert  ratings.  Based  on 
these  results,  items  were  divided  among  the  previously  stated  constructs. 

Finally,  each  of  these  items  were  given  to  the  students  in  two-way  audio/video  classes.  The  students  were 
asked  to  rate  how  strongly  they  agreed  with  each  of  the  items.  Ratings  were  given  using  a six  point  rating  scale  with 
one  being  strongly  disagree  and  six  strongly  agree.  Item  analysis  was  performed  using  coefficient  alphas.  From 
this  analysis  3 additional  items  were  removed  that  did  not,  contribute  additionally  to  the  measurement  of  the 
construct  in  question  and  detracted  from  the  overall  homogeneity  of  the  measure.  From  these  procedures  five  scales 
were  developed  to  assess  each  of  the  constructs  in  question. 

The  measures  included  items  such  as;  “Talking  on  the  camera  frightens  me”  (Anxiety:  5 items);  “I  would 
take  another  two-way  audio/video  class”  (Satisfaction  with  learning  experience  [SLEj:  6 items);  “The  lighting  in 
the  room  is  good”  (Physical  Environment  [PE]:  7 items);  “The  layout  of  the  room  makes  interacting  with  people  at 
my  site  easy”  (Physical  Layout  [PL]:  4 items);  “The  technology  sup;  >rt  for  this  course  is  good”  (Management  of  the 
two-way  audio/video  classroom  [MTAV]:  6 items).  Reliability  analysis  was  performed  on  each  of  the  constructs. 
Each  measure  was  found  to  have  acceptable  internal  consistency  (Anxiety  = 0.87;  SLE  = 0.90;  PE  = 0,77;  PL  = 
0.77;  MTAV  = 0.71).  Final  scores  for  each  of  the  measures  was  obtained  by  adding  each  of  the  items  for  the  specific 
construct  and  then  obtaining  the  mean. 

Procedures 

The  data  collection  took  place  over  two  semesters.  Nine  classes  participated  during  the  spring  semester  of 
1 997,  and  3 classes  participated  during  the  fall  semester  of  1997.  During  the  13th  week  of  class,  a questionnaire  was 
administered  to  each  participant.  The  questionnaires  consisted  of  several  items  assessing  a variety  of  topics 
including  the  anxiety,  SLE,  PL,  PE,  and  MTAV  constructs.  Before  receiving  this  questionnaire  the  participants 
received  an  informed  consent  form,  and  they  were  asked  to  participate  voluntarily  in  the  study.  Standard  human 
subject  procedures  were  followed  as  designated  by  each  school’s  human  subjects  committee. 
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Analysis  and  Results 

Scores  on  each  of  the  five  measures  in  question  (anxiety,  SLH,  PE,  PL,  MTAV)  were  obtained  for  each  of  the 
students.  Bivariate  correlations  were  run  on  these  scores.  All  of  the  measures  were  significantly  correlated  to  alpha 
= .01  level.  The  results  of  the  correlation  are  displayed  in  tabic  1 . 

Table  1.  Bivariate  correlations  for  anxiety,  satisfaction  with  the  learning  experience,  perception  of  the  physical 
elements  in  the  environment,  perception  of  the  physical  layout,  and  perception  of  the  management  of  the  two- 
way  audio/video  classroom . 


Anxiety 

SLE 

PE 

PL 

MTAV 

Anxiety 

1.0 

SLE 

.344** 

1.0 

PE 

322** 

.601** 

1.0 

PL 

.341** 

.647** 

.674** 

1.0 

MTAV 

.350** 

.658** 

.634** 

.581** 

1.0 

(**  Correlation  is  significant  at  .01 , two-tailed.) 


Next  step-wise  multiple  regression  analyses  were  performed  to  examine  the  contribution  of  the  different  physical 
environment  aspects  (PE,  PL,  MATV)  to  the  constructs  in  question:  anxiety  and  satisfaction  with  the  learning 
environment.  The  results  of  the  regression  analyses  are  displayed  in  table  2. 

Table  2.  Step-wise  multiple  regression  coefficients  (standardized)  for  perception  of  the  management  of  the 
two-way  audio/video  classroom  (MTAV),  perception  of  the  physical  elements  in  the  environment  (PL)  and 
perception  of  the  physical  elements  in  the  environment  (PE).  Anxiety’ and  satisfaction  with  the  learning 
experience  are  the  dependent  variables. 


Variable 

Anxiety 

Satisfaction 

MTAV 

.229** 

.376*** 

PL 

.208** 

.336*** 

PE 

.136* 

R squared 

.151** 

.547** 

* p < .05 
**  pc. 01 

***p  < .001 


The  results  of  the  anxiety  regression  analysis  (Table  2)  indicate  that  together  MTAV  and  PL  explain  a 
significant  portion  of  the  variance  in  students’  anxiety  with  an  R squared  value  of  .151  (with  p < .0! ).  Additionally, 
MTAV  and  PL  individually  explain  a significant  amount  (p  < .01)  of  the  variance  in  students’  anxiety.  The  step- 
wise procedure  excluded  the  independent  variable  PE  in  the  anxiety  regression  analysis.  The  exclusion  of  this 
variable  may  be  due  to  multicollincarity  among  the  independent  variables. 

The  satisfaction  regression  analysis  (Table  2)  indicates  that  together  the  three  independent  variables: 
MTAV,  PL  and  PE  explain  a significant  portion  of  the  variance  in  students’  satisfaction  with  the  learning  experience 
with  a R squared  value  of  .547  (with  p < .01.)  Individually,  each  of  the  three  independent  variables:  MTAV,  PL 
and  PE  explain  a significant  amount  (p  < .001 ) of  the  variance  in  students’  satisfaction  with  the  learning  experience. 
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Discussion 

In  this  study,  focus  was  placed  on  a few  of  the  foundational  elements  of  a two-way  audio/video  classroom. 
The  results  suggest  that  ignoring  these  elements  can  lead  to  a shaky  foundation  that  will  ultimately  affect  student 
anxiety  and  satisfaction  with  learning.  In  general,  the  results  of  the  bivariate  correlation  indicate  that  there  is  indeed 
a significant  positive  relationship  between  students’  perception  of  environmental  aspects  (physical  dements,  layout 
and  management  of  the  distance  learning  experience)  and  students’  anxiety  level  and  their  satisfaction  with  the 
learning  experience.  Furthermore,  the  regression  analysis  provides  support  that  the  students’  perception  of  the 
physical  elements  in  the  environment  and  the  management  of  the  distance  learning  experience  explain  a significant 
portion  of  the  variance  of  the  students’  anxiety.  Additionally,  students’  perceptions  of  the  physical  environment,  the 
layout  of  the  environment,  and  the  management  of  the  distance  learning  experience  explain  a significant  portion  of 
the  variance  of  students’  satisfaction  with  the  learning  experience.  Last,  there  are  some  issues  regarding  our  results 
that  need  to  be  investigated  further,  which  we  will  discuss  in  detail  below. 

The  items  within  each  of  the  scales  we  developed,  at  face  value,  appear  to  be  measuring  different,  but 
related  constructs.  However,  as  can  be  seen  by  the  correlation  analysis,  they  hold  a very  high  relationship  among 
themselves.  In  respect  to  the  environmental  perception  sub-scales,  caution  should  be  exercised  when  distinguishing 
one  of  these  specific  aspects  from  another.  This  problem  can  be  seen  in  the  first  regression  analysis  in  which  the  PE 
variable  was  excluded.  The  fact  that  it  did  not  significantly  contribute  to  the  explained  variance  of  anxiety  could  be 
due  to  multicollinearity  effects.  It  is  important  to  note  that  the  multicol linearity  may  not  be  the  fault  of  the 
measures.  It  seems  quite  possible,  given  the  distinction  novices  and  experts  attributed  to  the  question  in  the 
measurement  construction  phase  and  the  quite  opposite  finding  when  administering  it  to  distance  education  students, 
that  people  simply  do  attribute  different  values  to  these  distinct  constructs.  In  other  words,  students  may  simply  be 
answering  all  of  the  items  in  one  general  direction  or  the  other  and  not  making  any  finer  distinctions.  To  better 
assess  the  environmental  impact  on  student  anxiety  and  satisfaction,  a controlled  experiment  in  which  the 
environment  was  manipulated  would  need  to  be  done.  Yet  despite  these  problems  with  the  measures,  they  do  hold 
promise  for  explaining  overall  perceptions  of  environmental  aspects  as  well  as  a good  beginning  for  further 
refinement  in  the  measurement  of  specific  environmental  aspects  in  a two-way  audio/vidco  classroom. 

As  originally  hypothesized,  student  anxiety  and  satisfaction  arc  indeed  related  to  their  perception  of 
environmental  variables.  This  study  provides  the  first  steps  in  determining  what  can  be  done  to  help  alleviate 
anxiety  in  a situation  that  for  many  is,  by  nature,  anxiety  provoking.  In  addition  it  provides  yet  another  way  to 
address  sludent  retention  and  ensure  that  future  enrollments  continue.  Although  the  relationship  appears  to  be  fairly 
strong,  the  study  leaves  several  questions  for  further  exploration. 

Further  Research 

The  number  of  empirically  validated  measures  that  assess  the  environmental  aspects  of  a two-way 
audio/video  environment  arc  few  (Bincr,  Dean,  & Mcllingcr,  1994).  In  fact  we  could  not  find  any  measures  that 
specifically  addressed  this  question.  Thus,  in  the  first  portion  of  our  study  we  sought  to  establish  measures  that 
could  assess  each  of  the  various  elements  in  the  two-way  audio/video  environment  (physical  elements,  layout  and 
management  of  the  classroom),  students’  anxiety  level,  and  satisfaction  with  their  learning  experience,  A multi-step 
process  for  developing  these  measures  was  used.  Each  measure  had  an  acceptable  internal  consistency  level  and 
experts  as  well  as  novices  provided  ratings  that  indicated  a high  level  of  face  validity.  Although  this  is  an  important 
first  step,  additional  data  needs  to  be  gathered  so  that  validity  can  be  established  through  other  statistical  techniques 
such  as  confirmatory  factor  analysis.  Additional  support  for  validity  can  be  provided  through  an  examination  of  the 
relationship  between  measures  that  examine  the  same  or  similar  constructs.  Finally,  a controlled  experimental  study 
should  be  done  in  which  elements  in  the  physical  environment  are  held  constant.  This  will  then  assist  in 
determining  which  elements  in  the  environment  arc  influencing  students’  attitudes.  This  will  also  help  in 
determining  if  students’  attitudes  arc  influenced  by  their  perception  of  difierent,  distinct  elements  in  the  environment 
or  as  one  overriding  environmental  construct. 

The  study  sought  to  obtain  a measurement  of  students’  perceptions  of  the  environment.  This  was  deemed 
important  because  it  is  how  students  perceive  the  environment  that  is  most  likely  to  be  related  to  thcir  level  of 
anxiety.  However,  before  designers  can  make  blanket  decisions  about  how  to  create  a two-way  audio/vidco 
environment,  the  perception  of  environment  must  be  validated  against  actual  observations  of  the  environment  hy 
experts.  If  it  is  found  that  students’  perceptions  are  accurate,  this  can  lead  to  further  refinement  and  investigation 
into  which  aspects  of  the  environment  play  an  important  part,  in  student’s  anxiety  and/or  satisfaction.  Once  these 
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relationships  are  determined  further  studies  can  examine  if  different  layouts,  types  of  equipment  or  support  make  a 
significant  impaet  on  the  students’  anxiety  level  and  satisfaction. 

In  the  immediate  realm,  the  study  provides  evidence  that  the  foundational  elements  in  a two-way 
audio/vidco  classroom  - the  physical  elements  of  the  classroom,  the  design  of  the  classroom,  and  the  support  and 
management  - are  important  elements  in  regards  to  students’  anxiety  and  satisfaction  with  the  learning  experience. 
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Abstract 

In  this  article,  two  web  sites  developed  specifically  as  curriculum  resources  for  art  educators  are 
presented.  While  the  sites  were  not  designed  to  address  specific  principles  of  cognitive  flexibility  theory , we  make  a 
case  that  a thematic  approach  to  inquiry-based  art  education  delivered  in  a hypermedia  environment  designed  with 
three  layers  of  navigation  is  appropriate  for  an  ill-structured  domain  such  as  art.  An  early  response  from  teachers 
participating  in  an  on-line  seminar  suggests  that  the  complexities  of  inquiry  in  art  are  well  presented  in  a 
hypermedia  environment,  that  reductive  bias  is  minimized , and  that  this  environment  is  useful,  relevant,  and 
accessible.  This  article  focuses  on  the  development  of  the  sites  and  describes  a framework  for  three  blended  levels 
or  “ layers  ” of  navigation  that  support  knowledge  construction  within  the  domain:  near-linear,  guided,  and  self- 
directed  exploration.  Recommendations  for  application  of  this  framework  for  other  hypermedia  and  hypertext 
curriculum  resource  environments  conclude  the  article. 

Too  often,  web  sites  are  developed  for  instructional  uses  without  the  aid  of  sound  instructional  design 
principles.  These  sites  appear  on  lists  of  Internet  resources  for  teachers  and  can  confuse  novice  computer  users  and 
novice  teachers.  Content  is  presented  as  static,  verbal  information  pages  linked  to  other  information  pages  that  may 
or  may  not  include  obvious  or  intuitive  navigational  cues.  That  is,  information  is  delivered  with  the  new  technology 
but  the  format  of  the  presentation  may  confuse  or  “lose”  the  novice  and  intermediate  user. 

Designing  instruction  for  instructional  hypermedia  and  hypertext  requires  thoughtful  attention  to  certain 
aspects  of  learning,  particularly  when  designing  for  ill-structured  domains.  Over-simplification  of  the  complexities 
of  an  ill-structured  domain  encourages  novices  to  reduce  the  “solutions”  of  domain-specific  problems  to  simplified 
or  cook-book  answers,  which  is  known  as  reductive  bias  (Spiro,  Fcltovich,  & Co u Ison,  1992). 

In  ill-structured  domains  such  as  art,  an  over  simplification  of  the  domain  (e.g.,  the  reduction  of  learning  in 
art  to  production  skills  and  selected  historical  data),  masks  the  complexities  of  cultural  context  and  the 
interconnectedness  of  aesthetics,  criticism,  production,  and  history  in  the  art  world.  Similarly,  teacher  education 
programs  in  art  and  in-service  opportunities  for  novice  art  teachers  or  classroom  generalists  (elementary  educators); 
have  in  the  past  reduced  the  art  experience  to  a production  of  “happy  hands  and  holiday”  non-art  projects. 

In  this  article,  two  web  sites  (Erickson,  1996;  Erickson  & Cardenas,  1997)  developed  specifically  as 
curriculum  resources  for  art  educators  are  presented.  While  the  sites  were  not  originally  designed  to  address  specific 
principles  of  cognitive  flexibility  theory  (Jacobson,  1995;  (Spiro,  et  al.,  1992),  wc  make  a case  that  a thematic 
approach  to  inquiry-based  art  education  delivered  in  a hypermedia  environment  designed  with  three  layers  of 
navigation  is  appropriate  for  an  ill-structured  domain  such  as  art.  An  early  response  from  teachers  participating  in 
an  on-line  seminar  suggests  that  the  complexities  of  inquiry  in  art  arc  well  presented  in  a hypermedia  environment, 
that  reductive  bias  is  minimized,  and  that  this  environment  is  useful,  relevant,  and  accessible  (seminar  discussions 
may  be  found  at:  http://www.artsednet.gctty.edu/ArtsEdNetyhm/May97/0006.hlml).  This  article  focuses  on  the 
development  of  the  sites  and  describes  a framework  for  three  blended  levels  or  “layers”  of  navigation  that  support 
knowledge  construction  within  the  domain,  both  in  terms  of  new  content  knowledge  and  as  an  approach  for  novice 
teachers  to  use  to  teach  art.  Recommendations  for  application  of  this  framework  for  other  hypermedia  and 
hypertext  curriculum  resource  environments  conclude  the  article. 

Relevance,  Usefulness,  and  Accessibility  of  Hypermedia  Environments 

Much  has  been  written  about  the  educational  potential  of  hypermedia  environments  such  as  the  world  wide 
web  (WWW),  however,  not  all  web  sites  or  CD’s  are  educational  in  their  intent.  For  purposes  of  this  article,  wc  will 
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define  educational  hypermedia  as  those  environments  designer  and  developed  specifically  for  use  as  educational 
resources  or  as  a basis  for  instruction. 


Elements  of  Educational  Hypermedia  Environments 

The  relationship  between  the  site  and  the  user.  Successful  educational  and  instructional  hypermedia 
environments  are  more  than  the  information  presented.  That  is,  in  successful  educational  web  sites,  there  appears  to 
be  a synergistic  relationship  between  the  information  at  the  site,  the  authors  of  the  site,  and  the  users  of  the  site. 
Several  articles  have  identified  elements  of  this  synergy  (Barrett,  1992;  Carlson,  1991;  Corry,  Frick,  & Hansen, 
1997;  Hill  & Hannafin,  1997;  Jonassen,  1991;  Jonassen  & Wang,  1993;  Locatis,  Charuhas.  & Banvard,  1990; 
Schroeder,  1994;  Spiro,  et  al.,  1992;  Spiro,  Feltovich,  Jacobson,  & Coulson,  1991),  but  few  have  presented  them  as  a 
unified  whole.  Table  1 is  a first  attempt  at  illustrating  the  user  and  site  characteristics  designers  consider  when 
developing  successful  educational  web  sites. 

Table  L Elements  of  Hypermedia  Learning  Environments 


Leamer/Uscr  Elements 

Site  Elements 

Computer  Skills 

Content 

Content  Knowledge 

Presentation 

Self-efficacy 

Navigation 

Learner  Control 

Links  and  Nodes 

Task  type 

Hill  and  Hannafin  (1997)  recently  identified  several  lcarncr/user  elements  illustrated  in  Table  1:  Computer 
skills,  or  the  ability  to  use  the  computer  and  by  implication  a WWW  browser,  was  defined  as  system  knowledge  (p. 
39).  They  defined  content  knowledge  specifically  as  prior  subject  knowledge  (p.  39).  Such  prior  knowledge  is 
critical  to  the  acquisition  of  new  subject  or  content  knowledge.  Self-efficacy  “...refers  to  a personal  judgment  of 
one’s  capability  to  execute  actions  required  to  perform”  (p.  39).  In  other  words,  the  confidence  of  the  leamcr/uscr  in 
his  or  her  ability  to  use  the  computer  and  link  new  information  to  previously  learned  information  has  a great  impact 
on  how  one  uses  the  site  and  on  how  much  the  lcarncr/user  will  persevere  in  acquiring  new  information. 

Learner  control  is  another  element  that  has  been  discussed  at  length  in  designing  effective  instruction 
(Hooper,  Temiyakarn,  & Williams,  1993;  Malone  & Lepper,  1987;  Morrison,  Ross,  & Baldwin,  1991;  Ross  & 
Morrison,  1992)  and  is  of  particular  relevance  in  hypermedia  environments.  However,  because  of  the  complexity  of 
ill-structured  domains,  total  learner  control  may  not  be  an  effective  design  feature.  Cognitive  overload  may  occur  as 
learners  branch  further  and  further  from  the  site  in  search  of  content  knowledge  (Spiro,  et  al.,  1992). 

Related  to  learner  control  is  task  type.  The  type  of  task  a Icarner/uscr  has  in  which  a learner  engages  when 
entering  an  educational  hypermedia  environment  appears  to  determine  his  or  her  path  through  the  site  (Barab, 
Bowdish,  & Lawless,  1997).  From  an  art  education  perspective,  task  type  in  an  ill-structured  domain  such  as  art  is 
closely  tied  to  thematic  and  inquiry-based  learning  (discussed  below)  in  that  the  theme  and  line  of  inquiry  shape  the 
task. 

Previous  researchers  (Conklin,  1987;  Jacobson  & Spiro,  1995;  McLcllan,  1993;  Recker,  1994;  Schroeder, 
1994)  have  identified  site  elements  using  a variety  of  terms.  We  will  consider  the  list  in  Table  1 as  a basis  for  our 
discussion  of  site  versus  user  elements.  The  site  elements  Content,  Presentation,  Navigation,  and  Links  and  Nodes; 
refer  to  the  information  itself,  how  it  is  presented  to  the  learncr/uscr,  how  the  learner  moves  within  the  site,  and  how 
information  is  linked  to  other  information  respectively. 

While  either  set  of  elements  might  be  considered  separately,  hypermedia  design  appears  to  be  most 
successful  when  it  is  purposefully  designed  to  cross  lcarncr/user  elements  with  site  elements.  Though  most 
educational  hypermedia  environments  must  address  all  of  the  elements  in  some  degree,  our  focus  in  designing  the 
three  layers  of  navigation  is  primarily  on  task  type  and  navigation. 

Macro  and  micro  conditions  of  hypermedia  design.  In  addition  to  lcarncr/user  elements  and  site  elements, 
three  conditions  for  educational  hypermedia  appear  to  play  a role  in  the  success  or  failure  of  the  hypermedia 
environment  to  support  and  facilitate  learning:  relevance,  usefulness,  and  accessibility.  Each  condition  has  two 
levels  that  incorporate  the  larger  educational  context  of  the  user  (macro  level)  and  the  user  as  an  individual  (micro 
level).  Table  2 presents  these  conditions  related  specifically  to  art  education. 
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Table  2.  Macro  and  Micro  Levels  of  Relevance,  Usefulness,  and  Accessibility  of  Hypermedia  Learning 
Environments 


Condition 

Macro  Level 

Micro  Level 

Relevance 

Relevant  to  social,  institutional 
(school),  community,  and 
curricular  expectations. 

Personal  relevance  to  teacher  and 
learner:  the  environment  affords 
connections  to  prior  experience 
and/or  aids  in  delivery  of 
instruction  in  local  learning 
context  and  discipline. 

Usefulness 

Appropriate  presentation  of 
instructional  (and  artistic)  media 
for  structured  and  ill-structured 
domains. 

Use  of  the  hypermedia 
environment  reinforces  teaching 
and  learning  in  art  content  areas. 

Accessibility 

Socio-cultural  implications  of 
accessing  hypermedia:  race, 

gender,  socio-economic  status, 
availability  of  technology,  and 
the  ability  and  skills  needed  to 
use  technologies. 

User  interface  or  presentation: 
organization  of  content,  verbal 
and  non-verbal  presentation, 
graphic,  thematic,  or  concept  map 
navigation  within  the 

instructional  application 

facilitates  learning  and  minimizes 
disorientation. 

Relevance.  Relevance  refers  to  the  relationship  the  product  and  idea  technologies  have  to  the  user. 
Relevance,  in  the  sense  of  the  ARCS  model  of  motivation  theory  (Keller,  1979),  is  the  condition  that  mediated 
instruction  should  be  connected  to  the  teacher  and  learner's  personal  goals  and  values.  Relevant  web  sites  for 
instructional  needs  are  often  a matter  of  individual  perception,  but  may  well  be  a matter  of  availability  of  appropriate 
(relevant)  sources.  For  example,  graphic  software  is  highly  relevant  to  art  production  classes  but  may  be  less 
relevant  when  discussing  aesthetics. 

Usefulness.  Usefulness,  as  defined  for  this  article,  is  the  condition  that  relates  to  the  application  of 
different  hypermedia  environments  to  learning.  Usefulness  also  describes  using  the  technologies  to  the  appropriate 
level  of  their  potential,  and  in  the  appropriate  macro  and  micro  context.  The  technology  may  be  very  relevant  to  the 
instruction  (e.g.,  a downloaded  educational  game  in  art  history)  but  might  be  found  to  be  useless  in  terms  of 
learning:  the  game  is  fun  but  contributes  little  to  the  learning  context.  Usefulness  in  our  definition  should  not  be 
confused  with  usability  (Corry,  ct  al.,  1997). 

Accessibility.  Accessibility  on  a macro  level,  refers  to  the  social  and  cultural  constraints,  capabilities,  and 
implications  of  access  to  the  hypermedia  environment  and  to  the  technology  (Borrcll,  1992;  Piller,  1992), 
particularly  in  the  areas  of  race,  socio-economic  status,  and  gender.  On  a micro  or  local  level,  accessibility  is 
concerned  with  socio-cultural  issues  but  focuses  attention  on  individual  access  and  use  of  the  technology  and  the 
hypermedia  environment. 

Ill-structured  Domains  and  Cognitive  Flexibility 

Novices  in  ill-structured  domains  have  several  difficult  obstacles  to  overcome  before  they  are  welcomed 
into  the  culture  of  the  expert.  Medical  students  encounter  patients  whose  symptoms  do  not  conform  to  textbook 
examples,  budding  physicists  struggle  to  leap  from  ordered  word  problems  to  speculating  on  the  nature  of  motion, 
and  student  teachers  find  that  teaching  methods  are  seldom  performed  “by  the  book”  in  real  classrooms.  Novice  art 
educators  have  the  added  task  of  dealing  with  a content  area  that  is  so  ill-structured  that  it  is  often  assumed  to  lack 
substance  or  discipline. 

Ill-structured  domains  arc  those  which  may  be  characterized  as  having  two  distinct  and  interconnected 
properties:  “concept-  and  case-complexity " and  “across-casc  irregularity •”  (Spiro,  ct  al.,  1992,  p.  25).  Briefly  stated, 
concept  and  case  complexity  requires  that  learners  (students,  artists,  teachers)  hold  several  often  conflicting  concepts 
in  mind  when  confronting  a case  within  the  domain.  In  art,  cases  may  be  particular  problems  in  aesthetics,  history, 
criticism,  or  creating  a work  of  art.  For  example,  seeking  to  understand  an  artwork  from  another  culture,  such  as  a 
19th  century  Sioux  decorated  container  (art  history  problem)  may  require  one  to  hold  in  mind  such  diverse  concepts 
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as  climatic  conditions,  the  relationship  of  geometric  shapes,  and  pre-industrial  art  making  techniques  (Erickson, 
1996).  In  advanced  art  courses,  case  complexities  and  irregularities  such  as  these  increase  across  art  media  and  art 
concepts. 

The  irregularity  of  similar  cases,  that  is,  the  necessity  of  applying  different  concepts  and  actions  to  a case 
that  might  resemble  a previously  encountered  case,  is  often  encountered  in  art  and  may  indeed  play  a major  part  in 
how  art  movements  arc  started.  For  example,  an  aristocratic  portrait  from  Colonial  New  Spain;  a representation  of 
the  Mexican  Revolutionary  hero,  Zapata;  a mural  by  Diego  Rivera;  and  a feminist  self  portrait  as  the  Virgin  of 
Guadalupe  (Erickson  & Cdrdcnas,  1997)  are  examples  of  an  application  of  very  different  concepts  and  actions  to  the 
“common”  problem  case  of  painting  a portrait. 

The  two  web  sites  described  in  this  article  are  designed  to  guide  and  support  the  art  educator  in  developing 
curriculum  for  art  education  while  providing  some  sense  of  case  complexity  and  irregularity.  The  above  examples 
of  case  complexity  (seeking  to  understand  artwork  from  another  culture)  and  case  irregularity  (portrait  painting)  arc 
illustrations  drawn  from  the  two  web  sites.  In  turn,  the  lessons  and  supplements  presented  provide  the  K-12  art 
students  with  some  insight  and  a window  on  learning  in  art.  Well  designed  instructional  hypermedia  and  hypertext 
environments  should  offer  learners  rich  and  challenging  instructional  contexts  that  foster  learning  in  complex  or  ill- 
structured  domains.  Novice  learners  in  such  domains  as  art  or  medicine,  are  largely  struggling  to  understand  the 
complexities  of  the  domain  itself.  Add  to  this  the  necessity  of  learning  with  and  from  an  educational  technology  that 
affords  a non-linear  mode  of  accessing  information,  and  the  result  can  be  disorientation  and  cognitive  overload 
(Conklin,  1987;  Jacobson  & Spiro,  1995;  Spiro,  et  al.,  1992). 

By  overlaying  the  information  with  a thematic  and  inquiry-based  organizational  scaffold  and  through  the 
application  of  sound  instructional  design  elements,  much  of  the  confusion  and  disorientation  can  be  minimized 
without  sacrificing  the  presentation  of  the  complexities  of  the  domain.  We  identified  navigation  as  the  most  critical 
support  for  the  thematic  and  inquiry-based  approach  to  this  domain. 

Thematic  and  Inquiry-based  Learning 

In  “Our  Place  in  the  World”  (1996)  and  “Chicana  and  Chicano  Space”  (1997)  web  sites,  content  is 
organized  within  a thematic,  inquiry-based  approach  that  uses  a series  of  questions  to  help  students’  learning. 
Students  in  the  K-12  art  education  classroom  learn  to  formulate  additional  questions  and  search  for  answers  through 
their  own  art  experiences  and  by  researching  appropriate  sources.  Learning  in  a thematic,  inquiry-based  classroom 
is  student-centered  rather  than  teacher  led;  and  encourages  student  elaborations  on  the  themes  and  questions  to 
promote  deeper  processing  (Hannafin  & Carney,  1991 ; Wilson  & Cole,  1992). 

Our  Place  in  the  World  (1996)  focuses  on  four  key  inquiry  questions: 

1 . How  is  the  reproduction  different  from  the  original  artwork? 

2.  How  was  the  artwork  made? 

3.  What  visual  elements  did  the  maker  choose? 

4.  What  was  the  natural  world  like  where  the  artwork  was  made? 

Students  explore  art  making  questions  related  to  four  inquiry  questions  and  apply  the  inquiry  questions 
when  they  make  their  own  art  and  when  they  study  two  artworks  from  the  past  (an  Ice  Age  cave  painting  of  a bison 
and  a 19th  century  Sioux  parflcche  case).  The  theme,  Our  Place  in  the  World,  provides  an  overall  direction  for 
thinking  about  the  two  art  works  as  they  relate  to  each  student’s  new  experience  in  art. 

Chicana  and  Chicano  Space  (1997)  presents  another  inquiry-based,  thematic  unit  called  “Protest  and 
Persuasion.”  This  unit  focuses  on  the  following  key  inquiry  questions: 

1.  What  can  I learn  about  the  tools,  materials,  and  processes  the  artmakcr  chose? 

2.  What  can  I learn  about  how  the  patron,  user,  or  viewer  understood  the  artwork? 

3.  What  visual  elements  do  I see? 

Students  apply  the  three  inquiry  questions  to  art  making  as  they  plan  and  produce  their  own  print  or  mural 
and  as  they  investigate  the  twenty  Chicana/Chicano,  Mexican,  and  Mesoamcrican  artworks  posted  on  the  site.  The 
theme  of  Protest  and  Persuasion  provides  a focus  for  their  thinking  about  these  artworks  as  well  as  for  their  own  art 
making  experience. 

Thematic  instruction  from  a teacher’s  perspective  is  not  new.  Themes,  particularly  in  elementary 
classrooms,  provide  a useful  means  to  support  deeper  understanding  and  a context  for  learning.  In  ill-structured 
domains,  thematic  instruction  guides  students  through  the  learning  context  by  following  a theme  through  a variety  of 
situations  thus  promoting  an  experiential  learning  context  with  guidance  to  facilitate  learning.  Key  inquiry 
questions  help  students  understand  the  domain  and  can  introduce  new  levels  of  complexity  without  overwhelming 
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the  student.  In  this  way,  the  task  of  moving  through  the  site  is  organized  around  themes  and  inquiry  rather  than 
strictly  by  linear  step-by-step  links,  though  novice  teachers  may  navigate  the  site  in  a variety  of  ways. 

Layers  of  Navigation 

“Our  Place  in  the  World”  (OPITW)  and  “Chicana  and  Chicano  Space”  (CCS)  curriculum  resource  web 
sites  were  designed  for  teachers  to  use  rather  than  for  K-12  students.  Within  the  sites,  sets  of  lesson  plans  and 
suggested  paths  are  presented  that  help  the  teacher  develop  a thematic,  inquiry-based  approach  to  art  education. 
Beginning  and  experienced  art  teachers  often  have  a difficult  time  presenting  the  world  of  art  and  its  complexities  to 
their  students.  While  the  themes  and  questions  suggest  certain  paths  through  the  content,  navigational 
considerations  for  beginning  and  experienced  teachers  were  needed. 

As  the  prototype  for  OPITW  was  formed,  we  found  that  the  natural  flow  of  the  content  design  could 
become  too  linear  if  translated  directly  from  the  lesson  plans  to  a hypertext  or  hypermedia  environment,  even  though 
wc  had  the  option  of  hot  linking  terms  and  ideas.  When  these  materials  are  presented  in  a classroom  to  beginning 
teachers,  a large  amount  of  dialog  and  experimentation  with  themes  and  questions  becomes  the  focal  point  of  the 
class.  Our  vision  was  to  create  a site  that  could  provide  this  kind  of  interaction  with  the  content  and  yet  at  the  same 
lime  guide  the  novice  user  and  beginning  (eachcr  through  the  materials. 

To  accomplish  this  goal,  three  layers  of  navigation;  near-linear,  guided,  and  self-directed  exploration;  were 
designed  to  address  the  needs  of  novice  teachers  without  sacrificing  the  complexities  of  the  content.  These  layers 
roughly  parallel  the  three  levels  of  use  described  in  the  Jasper  Series  from  the  Cognition  and  Technology  Group  at 
Vanderbilt  (CTGV)  (1992). 

There,  through  a series  of  scenarios,  K-12  learners  are  presented  with  learning  tasks  that  arc  “anchored”  to 
"real  world"  problems;  an  approach  CTGV  described  as  anchored  instruction.  The  Jasper  Series,  as  envisioned  by 
the  CTGV,  allowed  three  levels  of  use;  each  level  progressively  less  dependent  on  direct  instruction.  However,  the 
Jasper  Series  materials  retain  many  of  the  characteristics  of  instruction  designed  with  traditional  ID  models:  the 
materials  tend  to  be  design  dependent,  follow  a somewhat  prescriptive  model,  and  are  intended  for  a large  audience, 
even  though  they  take  a more  constructivist  approach. 

The  levels  of  use  for  the  Jasper  Series  attempts  to  address  each  continuum  within  each  dimension.  The 
Basics  First  level  provides  a means  of  teaching  the  components  of  solving  Jasper  problems,  such  as  decimals, 
measurements,  time,  etc.  before  viewing  the  problem  scenarios.  Structured  Problem  Solving,  the  second  level  in  the 
Jasper  Scries,  provides  a means  of  introducing  complex  problems  with  simple  problems  to  help  students  avoid  error 
and  to  break  down  complex  problems  into  simple  components.  The  third  level,  Guided  Generation,  provides  a 
flexible  context  in  which  the  problem  is  approached  through  cooperative  groups,  some  guidance  by  the  teacher  as  a 
resource,  and  the  development  of  inquiry  skills  based  on  Vygotsky's  (Vygotsky,  1978)  concept  of  scaffolding. 

The  three  layers  of  navigation  in  OPITW,  while  based  on  the  model  of  CTGV’s  levels  of  use,  are 
specifically  designed  for  the  teacher  rather  than  for  their  studenls,  and  allow  greater  control  than  websites  with  only 
one  or  two  layers  of  navigational  strategics.  Novice  teachers  and  those  who  are  more  experienced,  may  move 
through  the  sites  using  three  distinct  navigation  paths. 

Near-linear  Navigation 

Near-linear  Navigation  may  be  defined  as  the  path  through  the  web  site  apparent  from  the  homepage.  That 
is,  the  OPITW  and  CCS  web  sites  imply  a path  from  Lesson  One  to  Lesson  Two  and  so  on  through  each  site,  rather 
like  the  Basics  First  approach.  Supplements  and  Extensions  in  OPITW  might  typically  be  accessed  after  all  of  the 
core  lessons  have  been  completed,  and  is  very  similar  to  a more  traditional  instructional  approach.  We  see  this 
strategy  layer  as  being  “near- linear”  in  the  sense  that  users  typically  follow  the  path  from  one  step  to  the  next  in 
sequence,  but  there  is  always  an  option  for  branching  to  other  links,  sections  of  the  site,  or  different  sites.  OPITW 
and  CCS  arc  not  closed  or  stand  alone  sites. 

Guided  Navigation 

The  Guided  Navigation  strategy  layer  allows  the  user  to  utilize  a set  of  outlines,  questions,  or  thematic 
paths  provided  on-line  to  structure  navigation  through  the  site.  To  further  support  Guided  Navigation,  a set  of 
simple  icons  was  created  to  graphically  represent  each  of  the  themes  and  sets  if  inquiry  questions.  Thus  teachers 
may  explore  themes  in  art  education  specifically  or  across  content  areas  as  an  integration  plan  by  individual  schools. 
Materials  included  in  OPITW  and  CCS  may  he  accessed  selectively  for  specific  applications  guided  by  the  needs 
and  parameters  of  the  integration  plan. 
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OPITW  and  CCS  include  a built  in  guided  navigation  strategy  featuring  the  use  of  graphic  icons.  Teachers 
may  click  on  an  icon  symbolizing  the  key  inquiry  questions  as  an  alternative,  less  linear  strategy  for  accessing 
curriculum  materials.  The  guided  questions  and  use  of  icons  were  a useful,  relevant,  and  accessible  means  of 
helping  novice  teachers  move  through  the  sites’  content  and  to  expand  upon  thematic  concepts. 

In  addition,  a second  guided  navigation  strategy  is  available  to  assist  teachers  in  considering  how  themes 
and  inquiry  can  improve  their  own  teaching.  An  on-line  seminar  provided  participating  teachers  with  reflective 
questions,  short  tasks,  and  guided  instruction  during  the  first  few  months  of  the  OPITW’s  availability  on  the 
Internet.  A listserv  moderated  by  the  second  author  was  put  in  place  to  provide  an  ongoing  dialog  with  an  expert 
while  teachers  used  the  site  in  their  own  classrooms.  In  this  way,  OPITW  actually  includes  two  types  of  guided 
navigation:  those  designed  into  the  site  as  icons,  themes,  and  inquiry  questions;  and  those  that  were  specifically 
asked  during  the  on-line  seminar  sessions. 

It  is  beyond  the  scope  of  this  paper  to  discuss  the  results  of  the  this  Guided  Navigation  layer  except  to 
mention  that  teachers’  response  to  the  listserv  and  tasks  were  positive  (sec  http://www.artsednet.  gctty.edu/ 
ArtsEdNet/hm/May97/0006.html).  In  many  cases,  this  layer  functions  as  another  “theme”  when  using  the  site  in  the 
sense  that  teachers  are  guided  by  the  tasks  or  needs  of  their  curriculum  plans.  Further  discussion  of  this  layer  will  be 
available  in  a later  article. 

Self-directed  Exploration 

“Our  Place  in  the  World”  and  “Chicana  and  Chicano  Space”  may  be  accessed  by  experienced  teachers  who 
require  the  site  to  serve  as  a curriculum  resource  rather  than  a tool  for  improving  their  teaching.  Ideas,  key  artworks, 
detailed  information,  inquiry  questions,  and  icons,  as  well  as  the  themes,  may  inspire  more  in-depth  exploration 
throughout  the  WWW.  In  fact,  “Protest  and  Persuasion”  on  CCS  concludes  with  a lesson  in  which  students  are 
asked  to  use  the  theme  to  guide  their  own  cruising  of  the  WWW.  OPITW  and  CCS  offer  a framework  for  lesson 
planning  and  curriculum  development,  yet  at  the  same  time  foster  further  exploration  of  the  domain. 

Recommendations 

Based  on  our  experiences  and  on  sound  instructional  design  practices,  we  suggest  that  designers  of  web 
sites  and  other  hypertext  or  hypermedia  environments  intended  as  educational  or  instructional  resources  for  ill- 
structured  domains  consider  the  following  set  of  considerations  as  a framework  for  good  design: 

1 . Consider  the  elements  of  hypermedia  environments  and  purposefully  design  for  “cross  over”  between 
learncr/user  and  site  elements.  By  striving  to  merge  site  and  user  characteristics  as  illustrated  in  Table  1,  designs 
for  educational  hypermedia  environments  become  more  useful,  relevant,  and  accessible  for  a variety  of  novice  to 
expert  users.  Sites  that  focus  only  on  one  set  of  elements  (learner/uscr  or  site  specific  elements)  may  be  appropriate 
for  presenting  verbal  information,  but  arc  not  useful  for  learning  in  ill-structurcd  domains  such  as  art. 

2.  Consider  the  segments  of  the  domain  to  be  presented.  Use  a thematic,  inquiry-based  approach  to  engage 
students  in  the  content  and  to  control  cognitive  overload.  The  sites  discussed  in  this  article  make  no  attempt  to  cover 
the  entire  world  of  art.  Rather,  they  guide  leamer/users  through  a series  of  strategics  for  moving  through  the  domain 
by  asking  questions  and  by  attending  to  suggested  themes.  Thus  the  domain  is  well  represented,  yet  cognitive 
overload  is  reduced. 

3.  Reduce  reductive  bias  hv  presenting  a variety  of  cases  with  open-ended  solutions.  Increase  complexity 
he  cases  by  presenting  alternative  extensions  to  the  cases.  Thematic  inquiry-based  learning  provides  a means  of 

presenting  cases  with  increasing  complexity.  By  encouraging  elaborations  and  curriculum  extensions,  case 
complexity  and  irregularity  can  be  addressed  as  typical  of  an  ill-structured  domain. 

4.  Use  a layers  of  navigation  approach  to  support  a variety  of  strategics  for  moving  through  the  site  and  to 
facilitate  learning  in  an  ill-structured  domain.  If  the  designer  pays  attention  to  the  interaction  of  site  and  user 
elements  rather  than  to  individual  elements,  the  design  has  a greater  chance  of  presenting  the  complexities  of  the 
domain  while  reducing  disorientation  and  reductive  bias. 

Further  experimentation  in  designing  layers  of  navigation  strategics  for  ill-structured  domains  is  needed. 
The  flexibility  of  instructional  hypermedia  and  hypertext  designs  provide  more  opportunities  to  address  higher  order 
thinking  in  ill-structurcd  domains.  By  attending  to  the  elements  of  instructional  hypermedia  environments 
(Icarner/uscr  and  site  specific)  and  by  meeting  the  conditions  of  relevance,  usefulness  and  accessibility;  designers 
can  meet  the  needs  of  learning  in  ill-structurcd  domains.  Developing  and  refining  navigation  strategics  for 
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instructional  hypermedia  such  as  the  three  layers  of  navigation  described  above,  will  aid  novices  and  experts  in 
understanding  the  complexities  of  ill-structured  domains  while  avoiding  reductive  bias. 
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Abstract 

This  paper  examines  research  studies  of  learner  control  in  computer-based  instruction  (CBI)  from  1987- 
1997 . Previous  literature  reviews  (Chung  &.  Reigeluth,  1992;  Ro "«■  &.  Morrison , 1989;  Steinberg,  1989;  Williams . 
1992)  have  drawn  predictions  and  recommendations  from  earlier  tamer  control  research.  This  review  evaluates 
these  earlier  conclusions  in  light  of  current  research , and  discusses  the  implications  of  recent  findings  for 
instructional  development. 

Learner  control,  defined  as  allowing  students  to  make  choices  among  instructional  events  during  a lesson 
(Schnackenberg  & Savenye,  1997)  has  been  investigated  with  learners  at  a variety  of  ages  and  grade  levels,  using 
lessons  covering  a wide  range  of  content  and  educational  outcomes.  The  idea  of  giving  learners  control  over 
elements  of  their  instruction  has  been  popular  among  educators  for  decades.  One  reason  often  advanced  for  learner 
control  is  that  learners  know  their  own  instructional  needs  best  and  therefore  are  uniquely  qualified  to  tailor 
instruction  to  these  needs  (Mager,  1964;  Merrill,  1975,  1980).  Previous  reviews,  while  encouraging  the  idea  of 
learner  control,  have  called  the  effectiveness  of  learner  control  “equivocal’*  (Williams,  1992),  “inconsistent,  but 
more  frequently  negative  than  positive”  (Ross  & Morrison,  1989),  and  “inconclusive,  and.. .more  frequently 
negative”  (Chung  & Reigeluth,  1992).  Researchers  have  continued  to  investigate  the  concept  of  learner  control, 
seeking  to  isolate  features  that  might  enhance  or  decrease  learner  performance. 

One  factor  affecting  learner  achievement  may  be  general  ability  or  aptitude  (Cronbach  & Snow,  1977). 
Williams  (1992)  points  out  that  some  studies  confound  general  ability  with  prior  knowledge.  Learner  ability  is 
associated  with  the  concept  of  general  intelligence  (Williams,  1992),  while  prior  knowledge  is  referred  to  as  the 
amount  of  information  a learner  has  previously  acquired  on  a particular  topic  or  subject.  Prior  knowledge  of  content 
is  generally  measured  through  a pretest  administered  before  assignment  to  an  instructional  treatment,  whereas 
general  ability  is  measured  via  standardized  tests  that  assess  general  aptitude.  In  the  studies  under  consideration  in 
this  review,  ability  measure  include  standardized  reading  tests  (Carrier  & Williams,  1988;  Kinzic,  Sullivan,  Beyard, 
Berdel,  & Haas,  1987);  Scholastic  Aptitude  Test  (SAT)  and  American  College  Testing  Assessment  (ACT)  (Schloss 
et.  al,  1988;  Schnackenberg  & Sullivan,  1997);  the  Henmon-Nelson  general  ability  test  (Klein  & Keller,  1990);  a 
figures  rotation  test  (McGrath,  1992);  and  a standardized  math  achievement  test  (Hannafin  & Sullivan,  1 995). 

The  use  of  standardized  test  scores  rather  than  prior  knowledge  examination  scores  in  the  aforementioned 
studies  enables  the  reported  achievement  results  to  be  attributed  directly  to  either  the  design  of  the  CBI  lesson 
utilized  or  general  student  ability,  and  not  to  the  information  and  skills  subjects  may  have  learned  prior  to  studying 
content  in  the  instructional  programs.  This  type  of  isolation  of  learner  knowledge  allows  the  specific  aspects  of  the 
Icarner-control/learner  ability  question  to  emerge  in  clearer  detail  and  not  be  mitigated  by  subjects'  content 
knowledge  in  a specific  area.  It  is  for  this  reason  that  this  review  focuses  only  on  studies  that  measure  learner  ability 
via  standardized  tests  because  it  is  only  in  this  way  that  accurate  conclusions  can  be  drawn  about  learner  control  and 
students’  general  intellectual  ability. 

It  has  been  suggested  that  lower-ability  learners  should  score  higher  on  posttests  under  program  control,  i.e. 
when  the  program  delivers  all  possible  instructional  events  in  a prescribed  sequence.  Chung  & Reigeluth  (1992) 
suggest  that  lower-achieving  learners  lack  the  knowledge  and  motivation  to  make  appropriate  decisions  regarding 
their  own  learning  needs,  making  learner  control  less  effective  for  them.  Steinberg  (1989)  makes  a similar  argument 
for  reserving  learner  control  for  higher-ability  students,  since  these  students’  mctacognitive  strategics  may  be 
presumed  to  be  better  developed. 
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To  investigate  a possible  interaction  between  ability  and  learner  control,  a number  of  recent  studies  have 
used  ability  measures  as  a blocking  factor  (Carrier  & Williams,  1988;  Hannafin  & Sullivan,  1995;  Klein  & Keller, 
1990;  Kinzie,  Sullivan,  Beyard,  Berdel,  & Haas;  1987;  Kinzie,  Sullivan,  & Berdel;  1988;  McGrath,  1992;  Schloss, 
Wisniewski,  & Cartwright,  1988;  Schnackenberg  & Sullivan,  1997).  None  of  these  studies  revealed  an  interaction 
between  ability  and  type  of  control.  The  lower-ability  students  did  not  perform  any  better  under  program  control 
(where  learners  could  not  choose  instructional  options)  than  under  learner  control.  However,  Kinzie  et.  al  (1987) 
did  find  differential  effects  between  eighth-grade  males  and  females:  a three-way  interaction  among  ability,  type  of 
control,  and  gender.  The  learner-control  condition  favored  high-ability  females  and  low-ability  males. 

Most  of  these  studies,  however,  found  extremely  strong  effects  for  general  ability  on  achievement.  Higher- 
ability  students  typically  did  much  better  than  lower-ability  students,  regardless  of  type  of  control.  This  effect 
occurred  among  sixth-  and  seventh-grade  students  with  CAI  lessons  on  advertising  concepts  (Carrier  & Williams, 
1988;  Klein  & Keller;  1990);  eighth-grade  students  learning  about  solar  energy  (Kinzie  et.  al,  1987;  Kinzie  ct.  al, 
1988);  ninth-  and  tenth-graders  in  geometry  (Hannafin  & Sullivan,  1995);  and  college  students  studying  special 
education  interventions  (Schloss  ct.  al,  1988)  and  instructional  design  principles  (Schnackenberg  & Sullivan,  1997). 

One  interesting  finding  emerging  from  several  of  the  studies  is  that  overall,  both  higher-  and  lower-ability 
students  using  learner  control  often  outperformed  students  under  program  control  (Carrier  & Williams,  1988; 
Hannafin  & Sullivan,  1995;  Kinzie  ct.  al,  1987;  Kinzie  et.  al,  1988).  However,  in  the  remaining  studies  (Hannafin  & 
Sullivan;  1996;  Klein  & Keller,  1990;  McGrath,  1992;  Schnackenberg  & Sullivan,  1997)  achievement  under  learner 
control  and  program  control  were  essentially  equal.  This  pattern  of  results  contradicts  findings  from  early  learner- 
control  studies  (Carrier,  Davidson,  & Williams,  1985;  Ross  & Rakow,  1981)  in  which  subjects  under  program 
control  often  outperformed  subjects  under  learner  control. 

In  light  of  results  reported  from  recent  research  done  in  the  area  of  learner  control  and  learner  ability,  the 
questions  of  whether  and  how  to  incorporate  learner  control  into  CBI  may  need  to  be  revisited.  The  posits  that 
higher-ability  students  score  better  under  learner  control  and  lower-ability  students  score  better  under  program 
control,  may  no  longer  hold  true  in  the  current  technological  age.  This  manifestation  may  be  due  in  part  to  the 
changing  nature  of  computers,  computer  software,  and  the  advent  of  the  Internet  which  occurred  in  the  timeframe 
between  earlier  learner  control/learner  ability  studies  and  more  current  research  in  this  area.  The  newer  forms  of 
media  have  made  learner  control  and  frec-access  navigation  more  common  features  encountered  by  computer-users 
of  all  ability  level'.,  thereby  perhaps  raising  the  overall  competence  in  the  appropriate  use  of  learner  control  for  many 
of  the  computer-using  population. 

As  new  forms  of  computer-based  instruction  arc  constantly  evolving,  continued  research  on  the  type  of 
control,  and  perhaps  on  the  amount  of  learner  control,  that  is  most  effective  with  learners  of  differing  ability  levels 
seems  justified.  Research  on  the  thought  processes  that  lower-  and  higher-ability  learners  use  to  make  learner 
control  decisions  may  also  yield  information  that  is  helpful  to  teachers,  instructional  designers,  and  researchers 
alike.  Perhaps  the  most  challenging  responsibility  of  future  research  in  the  area  of  learner  control  and  learner 
ability,  in  fact,  all  educational  technology  research,  is  in  the  to  attempt  to  keep  current  with  the  rapidly  changing 
nature  of  computer  technology  and  the  instructional  implications  that  emerge  from  these  new  innovations. 
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Abstract  of  the  Study 

Media  Sensor  is  a device  designed  to  sample  and  record  various  electrical  impulses  generated  during  a 
distance  education  session.  These  electrical  impulses  originate  from  a variety  of  visual  sources  at  both  the  near  and 
far  ends  of  a session,  as  well  as  audio  from  both  ends.  From  the  changes  of  voltage  recorded  by  the  Media  Sensor 
on  either  the  visual  or  audio  source , researchers  are  able  to  identify  specific  patterns  and  analyze  the  data  by  using 
a series  of  pattern  analysis  strategies  to  determine  whether  the  records  can  present  significant  events  occurred 
during  a session.  Accordingly,  both  the  instruction  and  the  classroom  management  of  an  instructor  can  be 
evaluated  by  the  results  of  the  pattern  analysis. 

In  this  study , the  research  team  defined  categories  of  classroom  events  by  analyzing  videotapes  of  several 
distance  education  sessions.  The  categories  are:  Teacher  Talking  (at  near  end),  Interaction  between  Far-end  and 
Near-end,  Interaction  at  Near-end,  and  Unknown/Other.  Furthermore , the  team  identified  patterns  by  comparing 
the  categories  and  the  raw  data  recorded  by  the  Media  Sensor,  testing  the  validity  and  reliability  of  the  patterns, 
and  applying  them  to  construct  the  context  within  distance  education. 

Purpose  of  the  Study 

The  purpose  of  this  study  was  to  explore  the  possibility  of  measuring  distance  education  classes  by  using 
the  Media  Sensor. 

Research  Questions 

The  research  questions  for  this  study  arc: 

• Can  the  Media  Sensor  identify  pattern  in  the  audio  and  video  data  recorded  from  distance  education  sessions? 

• How  to  identify  specific  types  of  patterns? 

• What  correlation  exists  between  the  pattern  and  the  actual  instructional  situation? 

• How  to  predict  instructional  events  by  identifying  specific  pattern  from  data  that  the  Media  Sensor  generates? 
Is  it  applicable  to  other  types  of  distance  education  session? 

Literature  Review 

Distance  education  is  categorized  into  three  generations  according  to  Kaufman  (cited  in  Bates,  1994,  a 
single  technology,  multimedia,  and  two-way  interactive  technology.  The  first  generation,  single  media,  is  mainly 
conducted  by  postal  service  based  on  paper.  There  is  no  interaction  between  students  and  the  instructor.  The  second 
generation,  multimedia,  is  represented  by  open  university  in  UK.  It  is  widely  distributed,  but  still  lacks  interactions. 
The  third  generation,  two-way  interactive  technology,  includes  video-conferencing  and  computer-mediated 
communication,  and  allows  interactive  communications.  In  the  present  paper,  we  will  discuss  distance  education 
categorized  as  the  third  generation.  One  of  the  major  concerns  in  distance  education  is  on  the  learner,  and  on  how 
instruction  can  effectively  support  or  facilitate  learning  (Moore,  1990).  There  are  various  studies  on  different 
aspects  of  distance  education.  Classroom  instruction  process,  students’  satisfaction  towards  distance  education,  and 
interaction  in  distance  education  arc  frequently  seen  in  distance  education  literature.  For  instance,  on  classroom 
instruction  process  and  classroom  management,  Westbury  and  Bel  lack  (1971)  identified  four  general  categories 
emerged  in  a classroom  process.  They  elaborated  on  "Teaching  Action"  as  one  of  the  major  instruction  events  in 
distance  education  sessions.  Further  more,  they  identified  three  teaching  actions:  “The  actions  of  a teacher  directed 
to  the  production  of  intellectual  acts  within  the  classroom,  such  as  teacher  talking,  or  making  the  students  to  talk; 
The  actions  of  a teacher  directed  at  making  the  students  ‘learningable’,  such  as  providing  motivational  factors;  The 
actions  of  a teacher  which  are  intended  to  contribute  directly  to  Ihc  students’  learning,  such  as  providing  practice  of 
learned  materials  or  techniques.”  (P.  243-245 ) 
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On  student  satisfaction  towards  instruction  in  distance  education  sessions,  Pugh  and  Siam?.  (1995)  found 
that  “student  satisfaction  did  improve  over  time/’  (P.  21)  They  also  reported  that  “evidence  from  the  observer 
comments  and  student  comments  tended  to  support  that  there  was  more  interaction  between  the  instructor  and 
students  and  between  students  at  the  near  end.”  (P.  22)  This  further  supports  that  interaction  is  a major  event  that 
might  lead  to  the  level  of  student  satisfaction  or  dissatisfaction. 

On  interaction  in  distance  education,  Moore  (1990)  presented  three  types  of  interaction:  learner-content 
interaction,  learner-instruction  interaction,  and  learner-  learner  interaction.  The  author  also  discussed  issues  such  as 
“what  level  of  interaction  is  essential  for  effective  learning,  what  is  good  interaction  and  how  to  achieve  it,  what  the 
real-time  interaction  contribute,  and  whether  it  is  worth  the  cost.”  Obviously,  various  types  of  interaction  in 
distance,  education  sessions  are  major  categories  of  events  that  this  study  should  identify  and  analyze. 

There  is  no  literature,  however,  on  pattern  analysis  on  the  distance  cducaiion  session,  and  no  study  has  been 
done  on  such  device  as  media  sensor  since  the  device  was  first  invented  by  Appelman.  The  study  intended  to  fill  in 
this  gap. 

A minor  concern  of  this  study  is  whether  the  tool  used  for  audio  and  visual  data  recording  --  the  Media 
Sensor  is  effective  in  recording  data.  Kounin  (1970)  in  his  study  on  group  management  in  classrooms  listed  out  the 
deficiency  of  using  human  observer  as  data-gathcring  medium.  This  finding  indirectly  supports  the  use  of 
mechanical  device  for  an  accurate  and  complete  data  analysis. 

Significance  of  the  Study 

Findings  from  the  study  can  be  applied  by  other  researchers  in  distance  education.  First,  this  study  intends 
to  reduce  the  time  of  observing  each  session  in  distance  education  by  establishing  a measurement  of  pattern  to 
identify  what  is  really  happening  in  the  session,  without  watching  the  video  tapes.  Second,  this  study  can  help  other 
researchers  to  expand  their  study  of  evaluating  instructor's  pedagogy  by  examining  outcomes,  teaching  styles,  and 
classroom  interaction.  Third,  this  study  provides  same  justification  of  the  cost-effectiveness  in  using  high- 
tcchnology  in  distance  education.  For  example,  if  there  are  only  few  interactions  between  far-end  and  near-end,  it 
seems  that  we  don’t  have  to  use  such  expensive  technologies  for  distance  education.  Instead,  wc  could  just  use  a 
VCR  to  record  the  class,  and  then  send  the  videotapes  to  students  at  remote  sites. 

Methodology 

This  study  is  to  develop  a measuring  tool  to  predict  what  happens  during  a session  in  distance  education 
without  watching  a video  tape.  The  Media  Sensor  invented  by  Dr.  Appelman,  a professor  at  Indiana  University, 
Bloomington,  generates  coded  data  sheet  from  video  taped  distance  education  sessions  with  time  coding  for  both 
audio  and  video  sources.  The  team  analyzed  the  data  to  find  specific  patterns  to  identify  significant  events. 

In  order  to  identify  the  pattern  of  the  audio  and  visual  data  collected  from  several  distance  education 
sessions  taking  place  at  the  School  of  Education,  Indiana  Univcrs5  Bloomington,  an  effective  and  efficient  method 
of  pattern  identification,  definition  and  analysis  needed  to  be  dec..^J.  Part  of  the  literature  review  was  conducted  in 
search  of  su  h a critical  tool  for  this  study.  Frick  recommended  the  method  of  “Analysis  of  patterns  in  time  (APT)” 
for  analyzin  observable  phenomena  so  that  the  patterns  of  these  phenomena  can  be  recognized.  APT  measures 
temporal  relations  between  variables  by  counting  their  occurrence.  Frick  found  out  that  using  proper  sampling 
strategies  could  lead  to  the  prediction  of  temporal  patterns  from  APT  results  (Frick,  1990). 

In  the  example  of  Classroom  Observational  Study,  Frick  used  APT  to  investigate  classroom  events.  “Highly  trained 
observers  collected  observational  data  on  paper-and-pencil  coding  forms.  For  illustration,  only  two  classifications 
arc  discussed:  available  instruction  (direct,  nondirect,  null),  and  student  orientation  to  academic  instruction 
(engaged,  nonengaged,  null).  . , .The  observers  also  coded  the  type  of  target  student  orientation  to  academic 
instruction  that  was  occurring  simultaneously  with  the  type  of  available  instruction.”  (1990,  p.  1 82) 

After  the  data  collection,  grouping,  and  the  calculation  of  its  means,  queries  were  made  about  these  APT 
scores  by  researchers  to  look  for  recurring  patterns  or  combinations  of  events  in  classroom.  Researchers  can  also 
aggregate  duration  of  certain  kinds  of  events  to  sec  what  proportion  of  the  overall  time  they  occupy.  This  data 
collection,  grouping,  and  analysis  method  provides  a foundation  for  the  methodology  of  this  study. 


358 


354 


Media  Sensor 

The  Media  Sensor  is  a device  that  can  capture  both  changes  of  brightness  on  a screen  and  voices  from  Far- 
end  and  Near-end  sites.  It  is  designed  to  sample  and  record  the  various  electrical  impulses  generated  during  a 
distance  education  session.  These  electrical  impulses  originate  from  a variety  of  video  and  audio  sources  at  both  the 
near  and  far  ends  of  the  session.  Video  sources  are  sampled  via  photoelectric  cells  placed  in  a grid  pattern  over  the 
screen  of  a television.  The  sensors  for  video  arc  located  in  five  areas  on  the  screen  (see  Figure  1 below).  Audio  is 
sampled  at  the  echo-cancel ler  as  "Far-end  Receive"  and  "Near-end  Send"  sources  (sensor  7 and  8 respectively, 
sensor  6 doesn’t  record  any  signal).  Figure  2 is  an  example  of  the  data  sheet  generated  by  Media  Sensor. 
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Figure  1:  Sensor  location  on  a TV  screen  (visual) 


Figure  2:  An  example  of  data  sheet 


Data 

Visual  and  audio  sources  pass  through  an  analogue  to  digital  converter  and  arc  sampled  at  various  rales  by 
the  Media  Sensor.  While  data  sampling  is  occurring,  records  of  each  sample  are  sent  to  a computer  as  ASCII  text 
for  recording  and  analysis.  Typical  patterns  of  a data  sheet  is  shown  at  the  end  of  this  Methodology  section  (Figure 
5). 
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Pattern  Definition  Process 

The  analysis  on  the  first  two  sessions  generated  the  first  version  of  the  pattern  definition.  The  team  tested 
the  accuracy  of  this  definition  in  the  analysis  of  the  next  two  sessions.  The  result  of  the  second  round  analysis 
generated  information  for  the  revision  of  the  first  pattern  definition. 

First  Pattern  Definition 

Two  video-taped  sessions  were  provided  to  the  team.  The  team  listed  several  events  that  occurred  in  class 
by  brainstorming  and  watching  a 30-  minute  segment  of  a session  on  the  videotape.  Then,  the  team  decided  how  to 
code  those  events  on  the  data  sheet,  as  shown  below. 

L = Lecturing 

T = Teacher  talking  (at  Near-end) 

FS  = Far-end  student  talking 
NS  = Near-end  student  talking 
N = Noise 

NA  = Lack  of  audio/voicc 
I = Interaction 

FS  ->  NS  (between  Far-end  students  and  Near-end  students) 

FS  ->  NS 

FS  ->  T (between  Far-end  students  and  teacher) 

NS  ->  NS 
NS->T 

M=  Camera  Movement 

T S (switch  between  teacher  and  students)  S ->  T 
P D (switch  between  people  and  document)  D P 

S S (switch  between  student  and  student)  S ->  S 

ZI,  ZO  (zoom  in,  zoom  out) 

Although  the  team  noticed  that  some  of  the  above  events  were  not  predictable  from  the  data  sheet  without 
knowing  the  instructional  content,  the  team  decided  to  observe  two  video  taped  sessions  and  code  the  entire  session 
according  to  those  categories.  Then  the  team  compared  the  coding  results.  The  two  persons  who  watched  the  same 
session  checked  with  each  other  to  detect  any  difference  in  the  coding. 

The  next  task  was  to  identify  patterns  in  order  to  predict  events  from  data  sheet.  The  team  looked  at  the 
data  sheet  vertically  from  sector  1 through  8 and  then  separated  voice  parts  (sector  7 and  8)  from  video  parts  (sector 
1 through  5). 

Throughout  the  entire  data  analysis  process,  the  team  defined  Noise  as  single  and  short  bars.  The  team 
understands  that  when  counting  the  amount  of  time  for  interaction  between  near  end  students  and  (he  teacher,  it  is 
possible  for  the  team  to  ignore  some  periods  of  teacher  talking  because  it  is  hard  to  tell  if  the  teacher  is  answering 
the  questions,  which  is  regarded  as  an  interaction,  or  starting  to  lecture  merely  from  looking  at  the  data  sheet. 

The  followings  are  our  patterns  for  four  categories,  lecture,  interaction  between  Near-end  and  Far-end, 
interaction  at  Near-end  and  Break. 

Lecture: 

Audio:  mostly  blank,  randomly  single  signal  appears  in  section  8; 

Signals  are  in  continuous  chunks. 

Visual:  More  signals  in  sector  4 and  5; 

Sector  1 has  almost  no  signal; 

Sector  2 has  signals  sporadically.  It  has  more  signals  than  sector  1 ; 

3 or  4 out  of  5 sectors  in  1-5  visual  sectors  have  signals. 

Interaction  (Near-  and  Far-end): 

Audio:  Signals  appear  in  pair  in  sector  7 and  sector  8. 

Visual:  Signals  appear  3 or  4 out  of  5 sectors  in  1-5  visual  sectors. 

Interaction  (Near-end  only) 

Audio:  More  blank  among  the  signals  in  sector  7; 

No  signal  in  sector  8. 
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Visual:  4 or  5 out  of  5 sectors  in  1-5  visual  sectors  have  signals  more  dense  than  those  in  sector  3 through 
5. 

Break: 

Visual:  More  dense  than  any  other  events  in  class  time. 

To  test  the  validity  of  the  patterns,  the  team  gained  two  more  video-taped  sessions  and  paired  up.  Each  pair 
looked  at  one  data  sheet  separately  without  watching  the  videotape,  and  compared  how  much  time  was  spent  for 
each  event  in  a session.  Then,  the  same  pair  looks  at  another  session  on  videotape,  coded  its  data  sheet  and  made 
comparison.  Therefore,  each  pair  conducted  two  analyses:  (1)  watched  a video  taped  session  and  coded  its  data 
sheet  during  the  observation;  (2)  coded  events  on  another  data  sheet  for  the  second  session  without  watching  the 
video  tape. 

The  team  also  decided  to  mark  colors  on  the  data  sheet  to  make  it  easy  to  differentiate  and  compare  each 
other's  coding  on  the  categories  of  events.  For  example,  pink  marker  was  used  for  interaction  among  Ncar-cnd 
students.  The  team  compared  the  coding  results  from  data  sheet  only  with  the  results  from  video  tape  observations 
to  see  if  any  agreement  (validity)  can  be  achieved.  Refer  to  Table  2 on  Page  18  for  the  results  of  video  tape 
observation  and  prediction  from  data  sheet. 

As  a result,  the  team  obtained  fairly  good  observer  agreement  on  the  total  amount  of  time  for  each  category 
calculated  by  persons  who  watched  the  video  tape  for  coding.  However,  the  team  had  a variety  of  results  between 
the  pair  partners  who  predicted  events  from  the  data  sheet  only,  and  between  the  pair  who  coded  only  the  data  sheet 
and  the  pair  who  coded  the  data  sheet  by  watching  the  videotape.  This  indicated  that  our  patterns  lack  strong 
reliability  and  validity. 

As  a result  of  the  data  analysis  at  this  phase,  the  team  obtained  advice  from  the  instructor  and  decided  to 
include  Unknown  as  a category.  The  team  also  realized  that  the  original  definitions  of  patterns  were  subjective  and 
vague.  The  team  redefined  the  patterns,  and  created  the  decision  making  tree  for  pattern  recognition.  Our  team 
obtained  a new  session  on  video  tape  taught  by  an  instructor  who  had  not  appeared  in  the  previous  sessions.  This 
time,  all  the  team  members  predicted  events  from  data  sheet  without  watching  the  video  tape,  compared  the  total 
amount  of  time  on  each  categories  of  events,  and  then  watched  the  tape  together  to  see  if  the  coding  results  from 
data  sheet  matched  what  actually  happened.  The  result  was  very  positive,  which  showed  to  the  team  that  the  new 
definition  of  pattern  was  applicable  at  least  to  the  analysis  of  this  session. 

Decision  Making  Tree 

The  team  developed  Decision  Making  Tree  to  present  the  pattern  definition  graphically.  It  can  also  be  used 
as  a method  for  data  interpretation.  The  first  Decision  Making  tree  was  developed  after  completing  the  first  round 
of  data  analysis  on  the  first  two  sessions.  After  the  data  analysis  on  the  next  two  sessions,  the  team  modified  the 
first  version  of  the  Decision  Making  Tree.  The  following  diagrams  are  the  first  and  second  versions  of  the  Decision 
Making  Tree. 
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Final  Pattern  Definition 

The  followings  are  our  final  definition  of  events  and  patterns: 

• Far-end  interaction:  dialogues  between  far-cnd  and  near-end  people. 

Patterns: 

( 1 ) In  Sector  8,  if  there  is  a group  of  signals  which  have  blanks  among  the  signals  that  last  longer  than  1 minute,  it  is 

a far  end  interaction. 

(2)  In  Sector  8,  occasionally  there  arc  individual  bars  whose  length  is  longer  than  1/3  of  the  maximum  length  of  a 

signal  bar.  These  individual  bars  are  also  regarded  as  far  end  interactions.  If  a blank  among  single  signals  or 
chunks  of  signals  is  longer  than  1 minute,  go  to  Sector  7.  If  a blank  in  Sector  7 is  smaller  than  1 minute,  it 
might  be  either  teacher  talking  or  near  end  interaction.  If  the  blank  is  equal  to  or  more  than  1 minute,  the  signal 
is  regarded  as  Unknown/Others,  which  can  be  break,  noise,  technical  problems  (e.g.,  with  sensors  or  recording 
devices,  etc.). 

• Near-end  interaction:  dialogues  between  people  at  near  end. 

Patterns: 

(1)  Ncar-cnd  interaction  starts  at  the  point  when  there  are  signals  in  each  sector  of  sector  1-5  simultaneously,  and 
ends  when  there  is  a signal  or  group  of  signals  starts  at  Sector  8,  which  is  a beginning  of  Far-cnd  interaction. 

(2)  Near-end  interaction  starts  at  the  point  when  there  arc  signals  in  each  sector  of  sector  1-5  simultaneously,  and 
ends  when  the  same  kinds  of  signals  through  sector  1-5  are  identified.  Thus,  the  interval  between  these  two 
signal  groups  is  regarded  as  the  length  of  this  near  end  interaction  time. 

• Teacher  talking:  teacher  is  either  talking  or  lecturing. 

Patterns:  If  it  is  not  a Ncar-cnd  Interaction,  and  there  are  signals  in  3 or  4 out  of  5 sectors  in  sector  1-5,  it  is  a 
Teacher  Talking  or  Lecturing. 
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Teacher  Talking 


Unknown/Others 


Research  Results 

According  to  the  definition  of  pattern,  the  team  studied  other  two  distance  education  program,  video  3 and 
video  4 (video  l & 2 were  the  first  two  video  sessions  studied  by  the  team).  The  results  are  shown  in  Table  1 and  2. 
D1  and  D2  arc  the  observers  who  predicted  the  instructional  events  by  only  looking  at  the  data  sheet;  DV1  and  DV2 
are  the  observers  who  worked  on  both  data  sheet  and  video.  The  figures  under  D1  , D2,  DV1  and  DV2  arc  the 
amount  of  time  each  category  of  instructional  event  lasted  in  these  two  sessions. 

Pearson  correlation  coefficient  is  calculated  within  a j lir  of  observers  and  between  the  two  pairs.  The 
coefficient  within  a pair  (D I and  D2,  DVl  and  DV2)  can  be  regarded  as  an  indicator  of  observer  agreement, 
coefficient  between  the  mean  of  the  Uvo  pairs  (the  mean  of  D!  and  D2  and  the  mean  of  DVl  and  DV2)  and  is  served 
as  an  indicator  of  validity. 
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D1 

D2 

DV1 

DV2 

Teacher  talking 

5'08" 

5745" 

20'40M 

2530" 

Interaction  (N) 

5' ib" 

14'30" 

1 TOO" 

8'30" 

Interaction  (F) 

14'20" 

39’ 15" 

6rooM 

58'00" 

Break 

0 

0 

11 '00" 

1 1'OO" 

Unknown 

86'52" 

0 

6’50" 

9'55M 

Observer  agreement 

-0.4176 
p < .484 

.9873 

p < .002 

Validity 

.1489  (pc. 81 1 1) 

Table  1:  Observation  on  video  3 (111.5  minutes) 

D 1 

D2 

DV1 

DV2 

Teacher  talking 

3 TOO” 

6 POO" 

47'3I" 

63'08" 

Interaction  (N) 

32,00,, 

* 



745" 

9'20" 

Interaction  (F) 

1 2'0O" 

1 P20" 

5 '3  5" 

6'42" 

Break 

0 

6 

3' 10" 

4'00‘ 

Unknown 

8"30" 

7' 10'' 

19'59" 

0 

Observer  agreement 

.5827 
P < .0303 

.8962 
p < .04 

Validity 

.947  (p  < .015) 

Table  2:  Obse nation  on  video  4 (84  minutes) 


The  observer  agreement  between  the  people  who  worked  only  on  data  sheet  separately  is  -0.4 176  (p  < .484) 
in  video  3 and  .5827  (p  < .303)  in  video  4,  which  means  there  is  very  little  agreement  when  the  two  observers 
predicted  instructional  events  by  applying  the  pattern  definition.  The  observer  agreement  between  the  people  who 
viewed  both  video  and  data  sheet  is  much  higher,  .9873  (P  < .002)  in  video  3,  and  .8962  (.04)  in  video  4,  which 
indicates  there  is  very  high  agreement  between  those  who  looked  at  both  video  and  data  sheet. 

The  validity  of  the  two  observation  is  different  from  each  other,  .1489  (p  < .81 11)  in  video  3,  and  .947  (p  < 
.015)  in  video  4.  The  validity  of  the  first  one  is  low  comparing  to  the  desired  correlation,  which  is  .60-. 70.  The 
second  one  is  extremely  high.  However,  according  to  the  observers’  comments,  the  defined  pattern  didn’t  work 
effectively,  because  there  was  so  much  uncertainty  when  the  observers  started  to  decided  which  category  an  event 
belonged  to.  Therefore  the  result  could  be  a random  guess.  Besides,  even  though  there  b very  good  match  between 
the  total  amount  of  time  of  each  category  of  instructional  events  as  predicated  by  D1/D2  and  traced  by  DV1/DV2, 
those  events  observed  by  different  observers  did  not  match  in  terms  of  starling  point  and  ending  point  of  time,  and 
often  times  these  were  totally  different.  Based  on  the  statistics  result  and  the  observer’s  comments,  the  team 
changed  the  specifications  of  pattern  definition.  The  team  then  applied  the  updated  definition  to  a new  distance 
education  session,  video  5 (Table  3). 
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Dl 

D2 

D3 

D4 

Teacher  talking 

8'00" 

25'40"" 

21,53n 

23'3~r 

Interaction  (N) 

1700" 

rooM 

2'40" 

1 ’08" 

Interaction  (F) 

52'40" 

523 1" 

52' 10" 

52' 5 7" 

Break 

0 

0 

_ ..  _ 

0 

0 

Unknown 

TOO" 

TOO" 

2'24" 

2' 10" 

Table  3:  Obser\>ation  on  video  5 ( SO  minutes) 


There  were  4 observers  working  on  video  5,  Dl,  D2,  D3,  and  D4.  All  observers  analyzed  the  data  sheet 
without  viewing  the  video  according  to  the  updated  definition  of  pattern  and  the  decision  making  tree.  Table  3 is  the 
result  of  this  analysis.  Table  4 is  the  Pearson  correlation  coefficient  between  any  two  decoding.  D2,  D3  and  D4 
have  very  high  correlation,  .9962  between  D2  and  D3,  .9986  between  D2  and  D4,  .9991  between  D3  and  D4.  The 
correlation  between  D1  and  the  rest  ones,  which  is  .8595  (DI  and  D2),  .8960  (D 1 and  D3)  and  .8768  (D1  and  D4) 
respectively,  is  not  as  high  as  those  between  any  two  of  D2,  D3  and  D4  because  of  some  misunderstanding  on  the 
pattern  definition  of  Near-end  Interaction  and  Teacher  Talking  between  this  observer  and  the  others.  This  explains 
why  there  is  bigger  difference  in  Teacher  Talking  time  and  Near-end  Interaction  time  between  Dl  and  the  rest  of  the 
observers.  Due  to  the  time  constraint,  the  team  did  not  count  the  amount  of  time  each  event  lasts  in  the  video  as  the 
team  did  in  video  4 and  video  3.  However,  (he  team  compared  the  analysis  result  with  the  video  and  found  out 
almost  all  the  events  in  video  matched  with  the  decoding  results  in  terms  of  the  categories  of  the  evens  and  starting 
point  and  ending  point  of  time  of  each  event. 


Dl 

D2 

D3 

D4 

Dl 

1 .000  (.000) 

D2 

.8595  (.062) 

1 .000  (.000) 

D3 

.8960  (.040) 

.9962  (.000) 

l.ood(.ooo) 

D4 

.8768  (.051) 

.9986  (".000) 

.9991  (.000) 

1 .000  (.000) 

Table  4:  Pearson  correlation  coefficient  between  pairs  of  observation  on  video  5 


Conclusion  /Discussion  /Limitation 

Conclusion 

The  team  only  checked  one  Distance  Education  session  using  the  latest  pattern  definition  and  analysis 
method  the  team  developed.  The  conclusion  can  be  only  applied  to  this  session.  It  is  concluded  from  this  study  that 
there  are  high  correlation  between  the  data  generated  by  the  Media  Sensor  and  the  instructional  situation.  The 
patterns  are  recognizable  from  the  data  sheet,  and  applicable  in  recognizing  the  instructional  events  without  seeing 
the  actual  session. 
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Limitations: 

• Validity  of  the  study: 

The  team  only  studied  five  sessions,  totally  8 hours  and  50  minutes.  Four  out  of  these  five  sessions  were  taught  by 
the  same  instructor  to  the  same  group  of  students.  The  definition  of  patterns  obtained  from  and  tested  in  these 
sessions  might  not  be  valid. 

• Generalization: 

Because  of  the  small  amount  of  sessions  the  team  studied,  and  the  unique  teaching  style  each  instructor  has,  the 
patterns  identified  from  these  sessions  might  not  be  applicable  to  other  sessions.  The  team  only  viewed  one  session 
after  the  final  revision  to  pattern  definition.  Thus,  the  conclusions  we  made  are  only  applicable  to  this  study. 

• Content- free: 

There  is  no  way  to  differentiate  the  tcchnical/equipment  testing  from  instruction  related  interaction  at  the  beginning 
of  each  session  and  interaction. 

• The  data  depends  on  the  experience  and  working  style  of  the  instructor  who  operates  the  camera.  Different 
instructors  could  have  different  styles  or  habit  when  they  operate  the  camera,  and  it  affects  the  signals  on  the 
data  sheet. 

• The  data  record  partly  depends  on  the  sensitivity  of  the  sensors.  There  are  some  differences  in  data  records 
when  rearranging  the  position  of  the  sensors. 

• It  is  impossible  to  differentiate  use  of  document  camera  and  near-end  interaction  because  both  of  them  arc 
accompanied  by  signals  in  all  5 video  sectors. 

• There  are  some  technical  problems  when  recording  the  signals. 

One  of  the  problems  the  team  encountered  was  that  it  was  uncertain  whether  the  Media  Sensor  was  reliable 
because  there  were  some  contradictions  between  the  video  tape  session  and  the  data  record.  Appclman  explained 
that  the  electronic  equipment  could  be  more  sensitive  than  human  eyes,  but  on  the  other  hand,  if  the  background 
remains  the  same,  media  sensor  can't  detect  the  change  even  the  camera  moved.  In  this  case,  it  happened  when  the 
team  did  not  see  any  movement  while  watching  the  video,  but  there  are  signals  on  the  data  sheet,  or  vice  versa. 

Discussion 

In  the  whole  process,  there  arc  two  stages:  coding  and  decoding,  as  shown  in  Figure  5.  Coding  happens 
when  patterns  were  defined  from  looking  at  the  video  and  using  the  data  sheet;  decoding  happens  when  applying 
these  patterns  to  predict  the  actual  situations  . In  either  of  coding  and  decoding  process,  some  information  is  lost 
inevitably.  There  is  also  some  distortion  between  the  predicted  patterns  and  the  actual  happenings  when  going 
through  the  coding  and  decoding  stage.  It  is  understandable  that  it  is  not  100  percent  accurate  to  apply  the  pattern 
definition  to  predict  the  instructional  events. 


Video 


Data  Sheet 


....  Predicted 

Pattern  : - Situation  . 


Figure  5:  coding/decoding  stages 


To  generate  more  accurate  and  applicable  pattern,  the  team  suggests: 

• Establish  a standard  of  camera  movement,  for  example,  the  camera  is  only  focusing  on  the  relevant  objects, 
with  only  the  speaker  appears  on  the  camera. 

• Revise  the  computer  program  depicting  the  data  with  smaller  scale  of  time. 

Future  Research 

Further  validity  studies  arc  needed  in  which  a wide  spectrum  of  distance  education  sessions  are  coded  by 
trained  observers  who  will  use  the  same  pattern  definition  and  pattern  recognition  method  by  viewing  the  data  sheet 
generated  by  the  Media  Sensor. 

If  such  validity  is  established,  then  the  next  step  would  be  to  see  if  the  APT  program  can  identify  the 
patterns  seen  by  human  eyes  when  looking  at  the  sessions.  The  APT  program  would  be  looking  at  some  form  of  the 
raw  data  produced  by  the  Media  Sensor. 

A third  step  is  to  specify,  define  and  identify  the  pattern  of  different  events  in  unknown  part,  such  as  break, 
noise,  etc. 

If  these  goals  can  be  achieved,  these  patterns  (from  Media  Sensor  and  APT  computer  program)  can  be 
correlated  with  other  measures  of  distance  education,  such  as  effectiveness  (c.g.,  student  learning  achievement), 
efficiency  (e.g.,  cost-benefit  analysis). 
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Abstract 

This  study  examined  the  effects  of  varying  instructional  strategies  on  learning  in  emerging  technology- 
based  visual  literacy  instruct  ion.  The  present  study  used  a completely  crossed  2 (organizational  strategy)  x 2 
(delivery'  strategy ) x 2 (management  strategy)  factorial  design,  post-test  only  control  group  design.  Organizational 
strategies  employed  included  criterion- referenced,  objectivist  instruction  and  problem-based , constructivist 
instruction.  Delivery  strategies  employed  included  local,  computer-based  instruction  and  networked  hypermedia- 
based  (WWW)  instruction.  Management  strategies  implemented  involved  participation  by  individuals  as  well  as 
cooperative  dyads. 

One  hundred  thirty-eight  undergraduate  education  students  were  randomly  assigned  to  eight  treatment 
groups  and  a control  group  and  received  self-paced  instructional  treatments.  Learners  were  assessed  on 
performance  ( achievement ),  instructional  efficiency , and  efficiency  of  learner  strategies  employed  (process 
efficiency). 

Achievement  measures  for  all  treatment  groups  showed  significantly  higher  levels  of  achievement  when 
compared  to  the  appended  control  group.  Using  multivariate  and  univariate  analyses  of  variance,  data  were 
examined  to  determine  what  effect  vary  ing  instructional  strategies  had  on  learning.  Results  indicated  no  statistically 
significant  differences  between  any  treatment  groups.  However,  some  strong  data  trends  were  observed.  Data 
trends  indicate  further  research  should  prove  worthwhile  in  determining  effects  of  instructional  strategies  on 
emerging  technology-based  instruction. 

Introduction 

Background  and  Theoretical  Perspectives 

Educators  today  debate  the  most  appropriate  instructional  roJc  of  technology,  particularly  computer 
technology.  The  past  ten  years  have  seen  the  development  of  two  trends  with  unprecedented  effects  on  the  course  of 
educational  technology:  an  increase  in  the  number  and  types  of  technology  resources  available,  and  dramatic  shifts 
in  beliefs  about  the  fundamental  goals  and  strategies  of  education  itself  (Robyler,  Edwards,  & Havriluk,  1997).  As 
the  goals  of  education  begin  to  change  to  reflect  new  social  and  education  il  needs,  teaching  strategies  also  change, 
and  so  do  strategies  for  integrating  technology  into  teaching  and  learning.  One  of  the  newer  technologies  available 
for  instruction  is  the  emerging  information  system  called  the  Internet  and  the  graphical-based  access  called  the 
World  Wide  Web. 

The  effectiveness  of  newer  interactive  technology  usage  in  instruction  is  open  to  debate.  Moore,  Myers, 
and  Burton  (1993)  state  that  little  actual  research  regarding  the  effectiveness  of  interactive  technologies  has  been 
conducted  and  what  has  been  conducted  has  iimited  value.  Practitioners  and  researchers  call  for  research  that  goes 
beyond  traditional  comparison  studies.  Grabowski  and  Pearson  (1988),  Sice  (1989),  Reeves  (1986),  Kozma  (1991), 
and  Moore  et  al(  1993)  have  made  calls  for  conducting  research  on  the  instructional  strategies  related  to  the  specific 
attributes  of  the  interactive  medium.  Hannafin  (1992)  identified  the  areas  of  constructivist  models  of  teaching  and 
acquiring  knowledge,  hypermedia,  and  cooperative 

teaching  and  learning  as  innovative  instructional  strategics  that  need  to  be  addressed  in  emerging  technology 
research. 

Instructional  Strategies 

Gagne,  Briggs,  and  Wager  (1992)  define  instructional  strategy  as  “...a  plan  for  assisting  the  learners  with 
their  study  efforts  for  each  performance  objective  (p.  27).”  Dick  and  Carey  (1996)  define  instructional  strategy  as  an 
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overall  plan  of  activities  to  achieve  an  instructional  goal.  Kettfp,  Morrison,  and  Ross  (1994)  differentiate 
instructional  strategy  from  delivery  strategy.  Instructional  strategy,  they  offer,  prescribes  sequences  and  methods  of 
instruction  to  achieve  an  objective,  and  delivery  strategy  describes  the  general  learning  environment.  Seels  and 
Richey  (1994)  define  instructional  strategies  as,  “...specifications  for  selecting  and  sequencing  events  and  activities 
within  a lesson”  (p.  31). 

In  contrast,  Reigeluth  ( 1 983),Merril  1 (1983),  and  Reigeluth  and  Merrill  (1978)  identify  three  components  or 
variables  of  instructional  strategies:  organizational  strategies,  delivery  strategies,  and  management  strategics. 
Although  this  instructional  strategy  lexicon  is  not  widely  used,  it  is  this  all-encompassing  view  of  instructional 
strategies  that  is  used  as  a framework  for  this  study,  as  indicated  in  Table  1,  below. 

Organizational  Strategies 

Organizational  strategies  are  those  decisions  the  instructional  designer  makes  when  designing  learning 
activities.  Most  important  of  these  decisions  is  how  the  designer  will  assist  learners  to  process  new  information  and 

to  process  at  a deeper  level, 
producing  meaningful  learning. 
Most  often,  this  is 
accomplished  by  the 
presentation  and  sequencing  of 
content.  How  content  will  be 
presented  and  sequenced  is 
most  often  determined  in 
response  to  what  type  of 
learning  is  to  take  place,  and 
how  the  designer  believes  an 
individual  learns.  Further 
examination  of  organizational 
strategies  requires  one  to  examine,  at  the  most  philosophical  level,  their  belief  whether  or  not  knowledge  exists 
independent  of  the  learner.  Which  epistemological  approach  to  learning  a designer  espouses  will  have  great  impact 
on  the  organizational  strategy  selected  for  use.  At  one  end  of  the  epistemological  continuum  is  objectivism,  while  at 
the  other  is  constructivism  (Jonassen,  1991a;  Jonassen,  Wilson,  Wang,  & Grabinger,  1993). 

Thede  (1995)  compared  constructivist  and  objectivist  frameworks  in  the  design  of  computer-assisted 
instruction  dealing  with  basic  chemistry  instruction  for  nursing  students.  The  objectivist  form  of  instruction  was 
loosely  criterion  referenced,  while  the  constructivist  form  was  problem-based,  but  in  a closed  environment.  Thcdc 
found  subjects  receiving  the  objectivist  treatment  scored  significantly  higher  than  the  constructivist/  problem-based 
group  on  objectivistic  post-test,  particularly  on  the  recall  level  questions.  The  objectivist  group  also  scored  higher  on 
comprehension  and  application  level  questions,  but  not  significantly  so.  The  treatments  varied  widely  in  their 
presentation  of  what  Thede  termed  identical  content.  In  addition,  no  constructivist  evaluation  was  given. 

Savery  and  Duffy  (1995)  consider  problem-based  learning  (PBL)  to  be  one  of  the  best  exemplars  of  a 
constructivist  learning  environment.  Very  little  research  has  yet  been  conducted  on  the  effectiveness  of  problem- 
based  learning  as  an  instructional  strategy  (Gallagher,  Stepian,  & Rosenthal,  1992).  Most  of  the  research  on  PBL  to 
date  has  focused  on  the  adjustment  of  faculty  to  an  innovative  technique,  (McAuley  & Woodward,  1984;  Moore- 
West  & O’Donnell,  1985;  Neamc,  1982),  student  perception  of  the  technique’s  usefulness  (Blumberg  & Daugherty, 
1989),  and  the  mechanisms  of  programmatic  change  (Thompson  & Williams,  1985).  Little,  if  any  research  has  been 
conducted  to  date  on  the  ways  in  which  problem-based  instruction  affects  student  learning  and  cognition. 

Many  studies  comparing  discovery  learning  and  expository  learning  have  been  reported.  Because  this 
comparison  approximates  somewhat  the  differences  between  constructivism  and  objectivism,  these  studies  bear 
review.  The  results  of  these  discovery-expository  comparison  studies  are  inconclusive.  Andrews  (1984),  Hall  and 
McCurdy  (1990),  Haukoos  and  Pcnick  (1987),  Lovelace  and  McKnight  (1980),  and  Sugruc  and  Thomas  (March, 
1989)  found  in  favor  of  discovery  learning.  In  other  studies,  no  significant  differences  were  found  by  Canino  and 
Cicchclli  (1988),  Elshout  and  Vccnman  (1992),  Gesi,  Massaro,  and  Cohen  (1992),  and  Haukoos  and  Penick  (1983). 

Delivery  Strategies 

In  the  design  of  instruction,  delivery  strategies  arc  those  decisions  affecting  the  way  the  information  will  he 
carried  to  the  learner.  Delivery  strategics  affect  the  selection  of  the  instructional  media  that  will  be  used  to  present 
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the  learning  activities  (Merrill,  1983).  This  study  focuses  on  two  delivery  strategies:  local  computer-based 

instruction,  and  as  indicated  by  Hannafin  (1992),  networked  hypermedia  based  instruction  on  the  World  Wide  Web. 

Computer-Based  Instruction 

Several  theories  have  been  the  basis  for  investigating  the  effect  of  computers  in  the  teaching  and  learning 
process,  including  behaviorism,  systems  theory,  and  cognitive  theory.  Most  of  the  techniques  applied  to  the  design 
and  use  of  CBI  can  be  traced  to  one  of  these  theories  (Simonson  & Thompson,  1997). 

In  an  attempt  to  design  effective  instruction,  early  media  research  compared  one  medium  with  another, 
typically  comparing  a new  medium  with  traditional  instruction.  This  type  of  research  focused  on  finding  the  ’’best” 
medium  of  instruction  for  teaching  students  regardless  of  the  learning  situation.  Overall,  this  research  represented  an 
effort  to  determine  which  medium  would  achieve  the  greatest  with  the  greatest  number  of  learners. 

Although  this  research  was  prevalent,  some  researchers  indicated  that  this  type  of  investigation  was 
inappropriate  as  a major  research  direction.  For  example,  Allen  (1959)  stated  that  enough  comparative  effectiveness 
research  had  already  been  conducted,  and  that  in  part,  research  should  focus  on  discovering  the  unique  educational 
characteristics  of  each  of  the  media  of  instruction.  He  reiterated  this  view  in  a later  article  (Allen,  1971)  adding  that 
such  research  should  also  include  the  relationship  of  media  characteristics  or  attributes  to  the  characteristics  of  the 
learner  and  the  nature  of  the  instructional  task.  A decade  later,  Clark  (1983)  also  made  a convincing  case  for 
research  that  emphasized  instructional  methods  incorporated  into  media  rather  than  studies  aimed  at  finding  the  best 
medium  for  instruction.  Instead  of  focusing  on  media,  Clark  (1975),  Levie  and  Dickie  (1973)  and  Saloman  (1974, 
1979)  suggested  that  we  study  “attributes”  of  media  and  their  influence  on  the  way  that  information  is  processed  in 
learning,  Clark  (1983)  explained  that  it  wasn’t  a particular  media  that  offered  advantages  to  the  learning  situation, 
but  more  a particular  attribute  of  that  medium  that  was  not  peculiar  to  only  that  medium,  but  shared  by  other  media. 

For  the  purpose  of  this  study,  this  researcher  has  identified  the  following  basic  attribute  categories  for 
computer-based  and  networked  hypermedia-based  instruction  (NHBI):  access,  content  presentation,  interactivity, 
and  locus  of  control.  Defining  design  attributes  of  CBI  and  NHBI  arc  displayed  in  Table  2,  below. 

Table  2.  Defining  Attributes  of  CBI  and  NHBI. 
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Networked  Hypermedia-Based  Instruction 

An  emerging  type  of  technology-based  instruction  is  networked  hypermedia- based  instruction  (NHBI), 
commonly  referred  to  as  Web-based  instruction.  Hypermedia-based  instruction  is  the  application  of  hypermedia 
techniques  to  educational  software  (Heller,  1990).  When  hypermedia-assisted  instruction  or  hypermedia-based 
instruction  is  delivered  over  a network,  or  more  specifically,  the  Internet,  it  is  called  networked  hypermedia-based 
instruction.  Networked  hypermedia-based  instruction  (NHBI)  is  referred  to  by  a variety  of  terms  and  abbreviations, 
including  WBT  (Web-based  training),  WBI  (Web-based  instruction),  and  IBT  (Internet-based  training)  (Driscoll, 
1997). 

NHBI  is  similar  to  traditional  multimedia  computer  programs.  In  NHBI,  learners  engage  in  self-paced 
learning  programs  that  use  multimedia  to  communicate  content.  Interactions  take  place  in  the  form  of  branching 
decisions  that  are  controlled  by  cither  the  learner  or  the  program.  These  programs  arc  most  frequently  used  to 
transfer  knowledge,  build  comprehension,  and  practice  applying  skills  (Driscoll,  1997).  The  simplest  form  of  NHBI 
programs  arc  electronic  books  that  feature  hypertext  and  images.  More  sophisticated  NHBI  programs  can  include 
audio,  video,  and  animation. 
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NHBI  differs  from  CBI  in  its  capacity  to  use  the  Internet  and  provide  communication  links  (Driscoll,  1997; 
Heller,  1990).  NHBI’s  greatest  advantage,  therefore,  is  access  to  the  multitude  of  resources  available  on  the  World 
Wide  Web.  The  node  and  link  structure  allows  that,  in  NHBI,  instruction  is  not  self-contained  into  one  application 
package.  Because  of  this,  more  than  one  document  may  be  open  at  one  time  as  the  content  may  be  spread  across 
many  network  sites. 

This  node  and  link  structure  is  one  defining  attribute  of  NHBI.  A hypertext  learning  environment  such  as 
NHBI  involves  semantic  nodes  which  are  linked  by  an  author  for  potential  learner  use.  The  semantic  nodes  are  often 
the  kind  of  word  that  might  be  indexed  in  an  ordinary  document.  Instead  of  being  listed  at  the  rear  of  the  document, 
such  words  are  linked  to  other  instances  or  other  associated  words  in  the  same  document  or  in  other  documents.  In 
NHBI,  these  links  can  take  the  learner  to  another  document  located  on  another  server  anywhere  in  the  world.  By 
pointing  and  clicking  on  such  words,  the  learner  can  travel  across  links  to  related  sources  of  information.  Other 
characteristics  that  distinguish  NHBI  from  its  precursor  CBI  are  given  in  Table  1 , above. 

Little  research  has  been  done  to  date  in  the  area  of  hypermedia  in  general.  Jonasscn  (1991b)  states,  “to 
date,  there  is  virtually  no  research  that  supports  or  rejects  semantically  structured  hypertexts”  (p.  93).  Frey  and 
Simonson  (1990)  report  a study  on  cognitive  style,  perceptual  modes,  anxiety,  attitude,  and  achievement  in  a 
hypermedia  environment.  However,  hypermedia  was  simply  used  as  a treatment  vehicle  and  its  relationship  to  other 
variables  was  not  examined.  Reeves  and  Harmon  (1989)  examined  the  relationship  between  a person’s  creativity 
and  hypermedia,  but  found  no  significant  relationship  between  the  two  variables.  Jonasscn  (1991c)  reports  a study 
exploring  graphical  browser  node  arrangement  in  the  form  of  an  expert’s  semantic  map.  A control  group  provided 
with  no  structural  information  regarding  nodes  reported  higher  recall  scores  than  treatment  groups  which  were 
provided  with  descriptions  of  the  nature  of  the  links  between  nodes.  The  results  of  the  study  seem  to  indicate  that 
hypermedia  may  not  be  as  effective  when  highly  structured. 

Management  Strategy/Task  Structure 

Management  strategies  are  those  decisions  made  in  the  design  of  instruction  that  affect  the  way  the 
individual  student  will  be  helped  to  interact  with  the  learning  activities  (Merrill,  1983).  They  arc  decisions  about 
which  organizational  and  delivery  strategy  components  to  use  during  the  instructional  process  (Reigeluth,  1983). 
Management  strategies  involve  motivational  techniques  and  individualization  schemes,  among  other  implementation 
activities  (Merrill,  1983).  How  learners  will  receive  instruction,  whether  by  lecture,  individual  scatwork,  in  small 
groups,  and  so  on,  is  an  example  of  management  strategy.  This  study  examines  two  management  strategics: 
individual  reception  of  instruction,  and  instruction  delivered  to  cooperative  groupings  of  two,  or  dyads. 

Individualized  Instruction 

Instruction  designed  to  meet  the  needs  of  the  individual  learner  has  long  been  a driving  force  in 
instructional  design.  Instructional  design  reminds  us  that  regardless  of  how  instruction  might  be  delivered,  be  it  to 
one  learner,  to  twenty  learners  or  to  400  learners,  learning  still  occurs  within  individuals.  Bloom  (1984)  maintains 
that  one-to-one  tutorial  instruction  is  the  most  effective  form  and  that  an  average  student  in  a tutorial  program 
achieves  more  than  98  percent  of  students  in  conventional  classroom  instruction.  The  history  of  instructional 
technology  also  indicates  a strong  commitment  to  individualized  instruction  which  accommodates  self-pacing, 
prescribes  instructional  methods  according  to  learner  characteristics,  and  permits  content  selection  according  to 
individual  goals  (Reiser,  1987).  The  rationale  for  individualization  in  technology-based  instruction  is  intuitively 
appealing:  people  differ  in  ability,  background,  readiness,  motivation,  interest,  and  learning  style  and  may  therefore 
require  different  types  and  different  paces  of  instruction  to  enhance  their  learning  (Hooper,  1992a).  Much  evidence 
supports  the  belief  that  optimum  learning  takes  place  when  the  learner  works  at  his  or  her  own  pace,  is  actively 
involved  in  the  learning  activities,  and  experiences  success  in  learning  (Kemp,  Morrison,  Sc  Ross,  1994). 

Cooperative  Learning 

In  contrast,  to  increase  the  occurrence  of  active  participation  on  the  part  of  the  learner.  Slavin  (1983) 
recommends  structuring  the  learning  environment  to  include  other  types  of  learner  interactions,  particularly  student- 
student  interaction,  so  as  to  provide  more  of  an  opportunity  for  students  to  take  an  active  part  in  their  learning. 
Cooperative  task  structures  refer  to  those  situations  when  group  of  two  or  more  individuals  can  or  must  work 
together  but  may  or  may  not  receive  rewards  based  on  their  group's  performance.  Virtually  all  cooperative  groups 
incorporate  a cooperative  task  structure  (Hooper,  1992a)  where  group  members  work  together  toward  a common 
task. 
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Research  on  cooperation  has  been  conducted  since  the  beginning  of  this  century  (see.  Mailer.  1929), 
although  classroom  research  on  practical  cooperative  methods  began  in  the  early  1970s.  Despite  the  many  studies 
done,  the  effects  of  cooperation  on  performance  are  still  rather  poorly  understood  (Slavin.  1983).  Johnson  ai  d 
Johnson  (1974),  in  a meta-analysis,  found  that  cooperation  is  better  than  individualization  or  competition  for  all  but 
the  most  concrete  and  repetitive  tasks.  Johnson,  Maruyama,  Johnson,  Nelson,  and  Skon  (1981)  found  strong 
evidence  supporting  cooperative  incentive  structures  over  individualistic  or  competitive  incentive  structures.  In 
contrast,  Michaels  (1977)  reviewed  much  of  the  same  literature  and  concluded  that  competition  is  usually  better  than 
cooperation  for  most  tasks. 

The  research  seems  more  conclusive  when  examining  cooperation  and  problem  solving.  As  early  as  the 
1930s,  Thorndike  (1938)  considered  the  superiority  of  group  to  individual  problem  solving  to  have  been  proved. 
Studies  that  examined  the  reasons  that  groups  did  better  on  problem-solving  tasks  concluded  that  they  did  better 
simply  because  of  the  pooled  problem-solving  abilities  of  their  members  (Slavin,  1983).  Faust  (1959),  Marquart 
(1955),  and  Ryack  (1965)  compared  groups  that  really  worked  together  to  groups  that  groups  that  were  randomly 
assigned  to  artificial  groups.  In  all  three  studies,  the  real  groups  had  higher  achievement  than  the  individuals,  but  not 
than  the  artificial  groups,  suggesting  that  the  real  groups  had  high  scores  not  because  of  interaction,  or  motivation, 
but  because  if  any  individual  in  the  group  could  solve  the  problem,  than  all  members  in  the  group  would  get  credit 
for  it,  regardless  of  their  participation  or  learning  (Slavin,  1983). 

Sufficient  interaction  is  also  a problem  in  computer-based  instruction.  Some  believe  that  individualized 
instruction  presented  by  computer  creates  “sterile’'  learning  environments  where  learners  interact  only  with  the 
computer,  apart  from  teachers  and  peers  (Hannafin,  Dalton,  & Hooper,  1987).  Dalton  (1990)  comments  that,  for 
many,  “individualized”  is  synonymous  with  “isolated.” 

Research  indicates  that  students  who  complete  CBI  in  cooperative  groups  generally  perform  as  well  as 
students  who  work  alone  (Carrier  & Sales,  1987;  Shlcchter,  1990)  and  often  perform  better  (Dalton,  Hooper,  & 
Hannafin,  1939;  Hooper,  1992b;  Johnson,  Johnson,  & Stanne,  1985,  1986).  Research  supports  cooperative  learning 
as  a means  to  improve  achievement  and  attitudes  in  computer-based  learning  environments.  Dalton  and  Hannafin 
(1987)  found  that  learners  working  in  pairs  significantly  outperformed  learners  working  individually  during  a CAI 
lesson.  In  addition,  learners  working  cooperatively  on  a CAI  lesson  produced  a greater  quality  and  quantity  of  daily 
work  and  demonstrated  greater  problem-solving  skills  than  learners  working  individually  or  pompetitively  (Johnson, 
Johnson,  & Stanne,  1985). 

Often,  the  effectiveness  of  cooperative  learning  is  attributed  to  interaction  among  group  members  (Webb, 
1989).  Therefore,  efforts  should  be  made  to  promote  interaction  in  cooperative  learning  situations.  Hooper  and 
Hannafin  (1991)  state  one  method  to  promote  interaction  involves  increasing  individual  accountability,  wherein 
each  group  member  must  demonstrate  mastery  of  content  in  the  instruction.  This  technique  might  also  be  used  to 
isolate  and  remediate  potential  learning  problems  within  the  group  and  to  provide  an  additional  incentive  to 
cooperate  (Hooper,  Ward,  Hannafin,  & Clark,  1989),  as  well  as  motivate  more  able  group  members  to  provide  help 
and  less  able  group  members  to  invest  sufficient  mental  effort  to  master  instruction  (Kerr,  1983;  Kerr  & Bruun, 
1983). 

Problem  Statement 

Hannafin’s  (1992)  call  for  research  for  examining  the  effects  of  varying  instructional  strategics  on 
technology-based  instruction  served  as  the  basis  for  this  study.  The  purpose  of  this  study  was  to  examine  the  effects 
of  varying  instructional  strategics  on  learning  in  technology-based  visual  literacy  instruction.  Specifically,  this 
study  examined  the  effects  of  instructional  organizational  strategics,  instructional  delivery  strategics,  and 
instructional  management  strategies  on  learner  performance  in  technology-based  visual  literacy  instruction. 
Interaction  effects  of  these  instructional  strategics  on  learner  performance  were  also  examined.  Learning  was 
assessed  in  three  forms:  achievement,  instructional  efficiency,  and  process  efficiency.  For  the  purpose  of  this  slu^g, 
instructional  efficiency  was  operationally  defined  as  the  level  of  achievement  (combined  achievement  score)  divided 
by  the  amount  of  time  spent  on  treatment.  Process  efficiency  was  opciationally  defined  as  a measure  of  the 
efficiency  of  the  process  of  learning  in  terms  of  learner  note-taking  strategies  employed  in  learning. 

Methods 

Subjects 

Subject*  in  the  present  study  consisted  of  138  volunteers  from  a pre-service  media  methods  course  at  a 
large  midwcslcrn  university.  The  course  is  a required  course  for  all  teacher  education  students  in  the  College  of 
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Education.  All  subjects  were  undergraduate  students,  primarily  sophomores  and  juniors.  They  ranged  in  age  from  19 
to  46  with  a mean  age  of  22.1  years.  The  gender  makeup  of  the  sample  population  was  34%  male  and  66%  female. 
In  addition,  subjects  represented  a variety  of  intended  major  areas  of  study. 

Materials 

Materials  consisted  of  four  separate  visual  literacy  instructional  packages  representing  two  levels  of 
instructional  organizational  strategics  and  two  levels  of  instructional  delivery  strategics:  a criterion  referenced  local 
CBI  lesson,  a problem-based  local  CBI  lesson,  a criterion  referenced  NHBI  lesson,  and  a problem-based  NHBI 
lesson.  Treatment  materials  were  developed  by  a design  team  consisting  of  one  CBI  designer,  one  NHBI  designer, 
and  two  content  advisers/subject  matter  experts,  one  of  which  was  the  investigator.  The  investigator  had  three  years 
teaching  experience  with  this  particular  subject  matter  in  the  course  which  served  as  the  subject  pool.  The  second 
content  adviscr/subject  matter  expert  was  a faculty  member  in  the  college  and  the  course  supervisor  with  15  years 
experience  in  the  field. 

Objectivist  Criterion  Referenced  Treatments 

The  criterion-referenced  modules  loosely  followed  Gagne’s  (1968.  1985)  recommended  events  of 
instruction.  Prior  to  the  commencement  of  the  lesson,  subjects  were  given  a one  page  introduction  to  the  lesson  that 
informed  them  that  there  would  later  be  a quiz  on  the  content  in  the  lesson.  An  introductory  section  attempted  to 
capture  the  learner's  attention  by  mentioning  and  showing  familiar  visual  images.  At  the  opening  of  each  section  of 
content,  subjects  were  informed  of  the  learning  objectives  for  that  section  on  screen.  The  information,  or  stimulus 
material  was  next  presented,  followed  the  learning  hierarchy.  7b  allow  learners  some  control,  some  sequencing  of 
content  was  permitted,  but  the  menu  selections  on  both  the  CBI  and  NHBI  versions  were  arranged  in  a suggested 
hierarchy  as  portrayed  by  a vertical  screen  menu.  When  faced  with  a vertical  menu,  subjects  tend  to  select  menu 
options  in  a top  to  bottom  order.  On-screen  questions  allowed  learners  to  practice  the  new  content.  General  feedback 
on  the  correct  response  was  provided  on  screen  for  learners. 

Constructivist , Problem-Based  Treatments 

The  problem-based  treatments  were  identical  in  content  to  the  criterion  referenced  treatments.  The 
differences  included  visual  portrayal  of  menu  selections  on  the  local  CBI  treatments,  no  advising  the  subject  of 
per-''  nance  objectives,  and  the  direction  the  subject  received  prior  to  commencing  the  lesson.  The  menu  screen  of 
the  local  CBI  problem-based  treatment  package  was  arranged  in  a chevron  shape  that  implied  no  inherent  order  of 
the  content.  This  was  done  to  encourage  the  subject  to  explore  topics  in  order  of  interest  or  need  to  solve  the 
problem  presented,  and  to  avoid  imposing  a hierarchical  order  of  content  on  the  learner.  Due  to  limited  options  for 
presenting  linked  lists,  the  NHBI  problem-based  version  of  the  module  used  a bulleted  menu  list  identical  to  that 
used  in  the  criterion-referenced  NHBI  module. 

Learners  were  not  informed  of  the  instructional  objectives  prior  to  the  presentation  of  content.  This  was 
done  to  allow  learners  to  construct  their  own  schemas  for  die  content,  rather  than  have  an  order  imposed  by  the 
learning  objectives.  Instead,  learners  were  presented  with  a teaching-related  problem  much  like  one  that  they  would 
face  in  the  real  world.  Learners  were  informed  that  they  must  create  a poster  and  an  overhead  transparency  to  help 
them  teach  an  instructional  design  topic  that  they  had  been  developing.  Learners  were  also  informed  that  they  would 
need  to  use  the  media  developed  in  a teaching  demonstration  to  their  peer  class.  It  war.  recommended  that  learners 
familiarize  themselves  with  visual  literacy  concepts  to  aid  them  in  their  media  development.  No  further  explanation 
was  given. 

Local  Computer-Based  Instructional  Treatments 

The  computer  -based  instructional  program  was  developed  using  Authorwarc®  on  Macintosh  personal 
computers.  Every  attempt  was  made  to  adhere  to  recommended  screen  design  guidelines  as  given  in  Alcssi  and 
Trollip  (1991 ) and  Schwier  and  Misanehuk  (1994). 

The  program  included  an  optional  introductory  section  on  how  to  navigate  through  the  program  with  the 
use  of  iconic  buttons.  This  section  was  made  optional  so  as  not  to  bore  the  more  technologically  competent  user. 
However,  the  design  team  felt  that  including  these  directions  might  facilitate  comfort  in  the  novice  user,  thereby 
focusing  their  attention  on  the  content,  ralhcr  than  the  distraction  of  operating  the  program. 

The  locus  of  control  rested  largely  with  the  program  in  terms  of  content  presentation.  Learners  were 
allowed  to  control  the  sequencing  of  content  through  menu  selection.  Once  a learner  was  within  a content  section, 
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however,  they  had  only  the  option  of  proceeding  to  the  next  screen  in  the  sequence,  backing-up  to  the  preceding 
screen,  or  returning  to  the  menu  screen. 

In  all  but  a few  instances,  the  learner  controlled  the  pace  of  screen  presentation.  However,  there  were 
instances  when  the  program  controlled  the  revelation  of  text  on  a screen  in  order  to  draw  the  learner’s  attention  to 
specific  points  within  that  section  of  content.  Text  was  never  removed  from  the  screen  without  the  learner  pressing 
the  forward  button;  instead  text  was  "dimmed"  on  revelation  screens  to  cue  the  learner  to  look  at  new,  appearing 
information. 

The  amount  of  content  presented  on  each  screen  was  kept  to  a limit  of  one  idea  or  content  “chunk”  per 
screen.  A.  total  of  218  screens  make  up  the  criterion  referenced  CBI  program,  while  212  screens  make  up  the 
problem-based  package  (less  screens  due  to  the  lack  of  objective  presentation).  In  order  to  keep  the  length  of  the 
progrr  .1  under  one  hour,  opportunities  for  learner  practice  were  limited.  At  the  end  of  several  sections,  however, 
learners  were  asked  thought  provoking  questions  in  text  and  visual  formats.  As  visual  literacy  is  a highly  visual 
topic,  an  *rrort  was  made  to  include  as  many  visual  representations  as  possible.  The  final  CBI  package  included  162 
scanned  in,.  clip  art  images,  and  hand-drawn  graphics.  All  images  and  artwork  included  in  the  program  were 
copyright-free.  All  content,  information,  and  practice  questions  were  contained  in  the  CBI  program.  No  outside 
materials  besides  the  introduction  page  were  made  available  to  the  students. 


Networked  Hypermedia-Based  WWW  Treatments 

The  networked  hypermedia  based  instructional  program  was  developed  using  identical  text  and  images 
from  the  CBI  version.  Text  and  images  were  copied  and  formatted  into  HTML  documents.  As  screen  design 
guidelines  for  Wcb-barxd  documents  are  lacking,  attempts  were  made  to  make  the  pages  as  user-fncndly  as 
possible. 

No  introductory  section  on  how  to  navigate  through  the  program  was  included,  as  learners  had  previously 
studied  navigation  on  the  WWW.  Learners  were  reminded  on  their  introduction  pages  that  links  were  presented  in 
an  alternate  color  to  ncn-linkcd  text,  and  that  clicking  on  a link  would  take  them  to  additional  information  on  the 
topic.  The  NHBI  version  of  the  lesson  consisted  of  21  separate  HTML  page  documents.  The  21  pages  included  53 
links:  nine  links  connecting  to  another  area  of  the  same  page;  38  links  connecting  to  a different  page  within  the 
lesson,  and  six  links  connecting  to  sites  external  to  the  lesson. 

The  locus  of  control  rested  largely  with  the  learner  in  terms  of  content  presentation.  Learners  were  allowed 
to  control  the  sequencing  of  content  through  a list  of  links.  The  Web  pages  were  planned  to  be  used  with  a browser 

such  as  Netscape  Navigator®  or  Microsoft  Explorer®,  that  in  addition  to  links,  would  allow  the  user  to  hack  up  to 
all  previously  viewed  screens  in  any  order. 

The  learner  controlled  the  pace  of  screen  presentation  through  the  use  of  scrolling  pages.  Revelation  was 
not  an  option  in  the  design  of  the  Web  pages,  and  so  was  not  included.  Text  was  never  removed  from  the  screen 
without  the  learner  scrolling.  For  most  of  the  sections  ol  content,  all  content  for  that  section  was  contained  in  one 
scrolling  page.  Sub-menu  level  lists  linked  the  user  to  separate  pages  for  design  elements  and  design  principles. 

The  final  NHBI  package  included  all  162  scanned  images,  clip  art  r :agcs,  and  hand-drawn  graphics  that 
were  included  in  the  local  CBI  package.  However,  many  of  the  images  were  reduced  to  smaller  sizes  so  as  to  reduce 
delays  caused  by  downloading  large  image  files  from  the  Internet. 

The  HTML  documents  were  loaded  onto  a Macintosh  server.  The  Web  pages  were  accessible  from  any 
type  of  personal  computer  with  Internet  access  (Macintosn  or  PC).  The  server  was  tested  to  ascertain  the  number  of 
users  or  “hits”  it  could  accommodate  at  one  time  It  was  determined  that  the  upper  limit  of  access  was  twelve 
simultaneous  users,  which  was  deemed  acceptable  for  the  size  of  the  treatment  groups  in  this  study.  For  the  purposes 
of  this  study,  the  Web  pages  were  placed  on  the  server  only  during  treatment  runs  (so  as  to  control  exposure  to 
subjects  assigned  to  NHBI  groups  with  Internet  access  outside  the  treatment  time  frame).  All  content,  information, 
and  practice  questions  were  contained  in  the  NHBI  program.  In  addition,  five  pointer  links  were  included  to  sites 
relating  to  visual  literacy  elsewhere  on  the  Internet.  Users  were  familiar  with  performing  searches  on  the  Internet,  so 
additional  information  could  be  sought  at  ihc  learner's  discretion. 

Dependent  Measures 

Learners  were  assessed  on  their  achievement,  instructional  efficiency,  and  process  efficiency.  Achievement 
for  this  study  was  a combined  score  of  a 30-item  objective  post-test  and  a 15-itcm  checklist  of  the  application  of 
visual  literacy  concepts  to  learner  generated  visual  media. 
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Achievement  Objective  Assessment 

The  achievement  posltest  was  designed  and  produced  by  the  researcher.  Based  cm  the  instructional  content 
of  the  visual  literacy  lesson,  the  instrument  measured  the  knowledge  gained  by  participants  during  their  use  of  the 
instructional  materials.  The  same  lest  was  gren  to  all  treatment  groups  and  to  the  appended  control  group.  The 
instrument  was  administered  to  participants  two  weeks  after  exposure  to  the  treatments  in  order  to  measure  long- 
term retention  of  knowledge. 

The  instrument  covered  all  sections  of  the  visual  literacy  content  and  was  composed  of  30  multiple  choice 
questions  with  four  possible  answers  to  each  question.  The  first  25  questions  were  entirely  text-based,  while  the 
remaining  five  questions  asked  the  learner  to  examine  or  interpret  a given  visual  in  order  to  generate  a response. 
Items  were  written  by  the  researcher  based  on  the  instructional  objectives  set  forth  in  the  instructional  design.  A 
typical  item  included  on  the  instrument  was:  “Age,  gender,  race,  and  past  experiences  arc  examples  of  a)  visual 
filters;  b)  socioeconomic  filters;  c)  cultural  filters;  d)  personal  filters.”  The  instrument  was  intended  to  serve  as  an 
object!  vistic  measure  of  achievement. 

The  content  validity  of  the  instrument  was  checked  through  the  inspection  of  qur.  lions  by  subject  matter 
experts  in  the  field  of  visual  literacy  who  were  asked  to  check  for  overall  content  and  accuracy.  Using  data  obtained 
during  the  study,  the  instrument  was  examined  for  internal  consistency.  After  examining  the  inlercorrclations  of 
item  score  and  total  score  on  the  objective  assessment  instrument,  six  questions  were  dropped  due  to  low  correlates. 
Reliability  of  the  remaining  22-item  showed  a Cron  bach’s  coefficient  alpha  of  0.65  (n  items  = 22). 

Achievement  Application  Assessment 

Jonasscn  (1992)  recommends  for  constructs, slic  learning  that  more  goal-free  evaluation  mi  aods  be  used 
and  that  it  is  higher  order  thinking  skills  that  should  be  assessed.  In  response  to  this,  a second  achievement 
assessment  instrument  was  designed  and  produced  by  the  researcher.  This  second  instrument,  was  designed  to 
measure  the  application  of  visual  literacy  concepts  and  design  principles,  rather  than  simple  recall.  This  application 
assessment  was  based  on  the  instructional  content  of  the  visual  literacy  lesson  to  measure  the  knowledge  gained  and 
applied  by  participants  after  their  use  of  the  instructional  materials.  The  same  assessment  was  performed  on  all 
treatment  groups  . Participants  were  given  two  weeks  after  treatment  to  produce  an  instructional  poster  and  an 
overhead  transparency.  The  instrument  assessed  participant  production  in  order  to  measure  application  of  visual 
literacy  knowledge. 

The  instrument  covered  all  sections  of  the  visual  literacy  content  and  was  composed  of  15  items  or 
categories  with  five  possible  ratings  in  each  category  (none,  poor,  moderate,  good,  and  excellent).  A typical  item  for 
rating  on  the  application  assessment  was:  “Apparent  eye  path  (use  of  COZULST  layout)”.  The  instrument  was 
intended  to  serve  as  a constructivist  measure  of  achievement. 

As  with  the  objective  assessment  of  achievement  instrument,  the  content  validity  of  the  application 
instrument  was  also  checked  through  the  inspection  of  categories  by  subject  matter  experts  in  the  field  of  visual 
literacy  who  were  asked  to  cheek  the  instrument  for  overall  content  and  accuracy.  Using  data  obtained  during  the 
study,  the  reliability  of  the  fifteen  item  constructivist  checklist  which  was  applied  to  the  subject-generated  poster  and 
overhead  transparency  yielded  a Cronbach’s  coefficient  alpha  of  0.95  (n  items  = 30). 

Combined  Achievement  Measure 

As  a variable  with  low  reliability  may  include  errors  of  measurement  which  may  attenuate  F ratios  and 
generally  confound  interpretations  of  experimental  effects,  Pru/.ck  (1971)  recommends  combining  highly  similar 
measures.  In  keeping  with  this  recommendation,  the  22-itcm  pencil  and  paper  objective  measure  and  the  two 
applications  (one  for  each  the  poster  and  the  overhead  transparency)  of  the  15-item  checklist  were  treated  as  a 
combined  achievement  score.  For  the  113  subject:',  the  internal  consistency  of  the  combined  achievement  measure 
calculated  a Cronbach  alpha  of  0.93  (n  items  = 52). 

In  st  ruction  a l Effici  ency 

Instructional  efficiency  was  defined  as  the  combined  achievement  post-test  scores  divided  by  the  lime 
required  to  complete  the  lesson.  Units  for  this  measure  were  given  as  score  per  minute. 
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Process  Efficiency 

Process  efficiency  was  operationally  defined  as  a measure  of  the  efficiency  of  the  learner  strategies  (note- 
taking)  employed  during  the  instruction.  Process  efficiency  is  calculated  as  the  total  achievement  score  multiplied  by 
time  spent  on  treatment  and  divided  by  the  quantity  of  notes  taken  (word  cou  .1). 

Procedure 

All  subjects  in  this  study  were  randomly  assigned  by  computer  to  one  of  eight  treatment  groups  as  part  of 
the  required  curriculum  of  the  course: 

• OLI:  Objectivist  (Criterion  referenced)  local  CBI,  individual; 

• OLC:  Objectivist  (Criterion  referenced)  local  CBI,  cooperative  grouping; 

• OWI:  Objectivist  (Criterion  referenced)  Web-bascd  (NHBI),  individual; 

• OLC:  Objectivist  (Criterion  referenced)  Wcb-bascd  (NHBI),  cooperative  grouping; 

• PLI:  Constructivist  (Problem-based)  local  CBI,  individual; 

• PLC:  Constructivist  (Problem-based)  local  CBI,  cooperative  grouping; 

• PWI:  Constructivist  (Problem-based)  Web-based  (NHBI),  individual;  and 

• PWC:  Constructivist  (Problem-based)  Wcb-based  (NHBI),  cooperative  grouping. 

Subjects  experienced  treatment  during  their  individual  class  meetings.  At  the  treatment  session,  subjects  were 
directed  to  their  assigned  computer  workstation,  asked  to  complete  a demographic  questionnaire,  and  to  read  the 
introduction  handout.  Both  the  CBI  package  and  the  NHBI  Web  site  were  pre-loaded  for  the  subject’s  convenience. 
Subjects  assigned  to  cooperative  groupings  were  directed  to  wait  for  their  assigned  partner  to  commence  the 
treatment. 

All  subjects  were  supplied  with  a notebook  in  which  to  record  notes  for  study  purposes.  Notebooks  were 
encouraged  as  generative  learning  aids  such  as  note  taking  and  outlining  require  the  learner  to  take  an  active  and 
responsible  role  in  the  instruction/learning  process  (Jonassen,  1988;  Wittrock,  1974).  The  notebook  was  intended  to 
promote  deeper  processing  of  the  content,  and  to  keep  subjects  from  simply  scrolling  through  the  lesson.  The  subject 
was  instructed  to  record  their  start  time  and  ending  time  on  the  cover  of  their  notebook.  Notebooks  were  collected 
after  treatment,  photocopied,  and  relumed  to  the  subject  in  the  next  class  period. 

Content  review  via  computer  required  between  30  and  100  minutes  and  was,  for  the  most  part,  completed 
during  the  regularly  schedule  50  minute  class  period.  Students  voluntarily  stayed  after  class  time  in  order  to 
complete  the  modules.  After  completion  of  the  instructional  treatment,  subjects  were  dismissed.  The  subjects  then 
had  two  weeks  in  which  to  produce  their  instructional  media.  Subject  notebooks  were  returned  so  that  they  could  use 
their  notes  for  reference  and  study  purposes.  The  treatment  materials  (CBI  and  NHBI  packages)  were  not  available 
for  reference. 

Two  weeks  after  treatment,  section  instructors  collected  subject-created  instructional  media.. At  that  time, 
each  subject  completed  the  30-item  objective  achievement  instrument  during  a regular  class  meeting.  Completion  of 
the  in-class  assessment  instrument  required  approximately  fifteen  minutes. 

Subject  instructional  media  productions  were  digitally  imaged  for  later  assessment  by  the  researcher  and 
returned  to  the  subjects  for  further  course  use.  The  researcher  reviewed  the  digitally  imaged  media  productions, 
using  the  application  achievement  instrument  to  assign  each  subject  and  application  achievement  score.  Scoring  of 
all  measures  was  performed  blind  to  treatment  group  assignment 

In  addition  to  the  138  subjects,  and  additional  15  subjects  volunteered  to  serve  as  an  appended  control  for 
the  study.  The  15  control  subjects  were  students  in  the  same  course  during  a later  semester.  The  appended  control 
group,  without  the  benefit  of  instruction,  completed  the  objective  measure  and  produced  both  a poster  and  an 
overhead  transparency.  The  application  measure  checklist  was  again  used  to  quantify  the  application  of  visual 
literacy  concepts  to  the  production  of  educational  media.  The  appended  control  group  served  as  test  of  the  reliability 
of  the  instructional  materials. 

Experimental  Design  and  Data  Analysis 

The  present  study  primarily  used  a completely  crossed  2 x 2 x 2 (2^)  factorial  design,  post-test  only  control 
group  design  with  organizational  strategy,  delivery  strategy,  and  management  strategy  as  the  independent  variables. 
The  first  independent  variable,  organizational  strategy,  had  two  levels  — a constructivist,  problem-based  approach  to 
instruction  and  an  objective  oriented,  criterion  referenced  approach  to  instruction.  The  second  factor,  delivery 
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strategy,  also  had  two  levels — via  local  computer-based  instruction  and  networked  hypermedia-based  instruction 
via  the  World  Wide  Web.  The  last  independent  variable,  management  strategy,  was  also  measured  at  two  levels  of 
individualization  schemes,  individual  subjects  and  subjects  in  cooperative  groupings  of  two. 

The  dependent  variables  for  this  study  included  a combined  score  of  two  measures  of  achievement,  a post- 
test,  and  an  evaluation  of  the  application  of  visual  literacy  principles  to  the  design  of  two  pieces  of  educational 
media  (a  poster  and  an  overhead  transparency).  In  addition,  instructional  efficiency  as  measured  by  L/.al 
achievement  divided  by  time  spent  on  treatment  was  a second  dependent  variable.  Process  efficiency,  a measure  of 
the  efficiency  of  the  learner  strategies  (note-taking)  employed  during  the  instruction,  was  a final  dependent  variable. 
Process  efficiency  is  calculated  as  the  total  achievement  score  multiplied  by  the  quantity  of  notes  taken  (word  count) 
and  divided  by  the  lime  spent  on  treatment.  The  factorial  matrix  used  is  portrayed  visually  in  Figure  1 . 
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Figure  1.  2x2x2  Factorial  Design  of  Independent  Variables 

The  relationship  between  dependent  variables  was  assessed  by  examining  correlations  between  dependent  variables 
and  by  examining  scatter  plots  of  dependent  variable  scores  to  assess  the  direction  of  any  relationships.  Preliminary 
data  analysis  for  this  study  was  accomplished  through  MANOVA  procedures.  MANOVA  effects  were  followed 
with  univariate  analysis  of  variance  procedures  to  determine  effects  of  and  interactive  effects  of  the  organizational 
strategics,  delivery  strategies,  and  management  strategics  on  each  of  the  three  dependent  variables. 

Results 

Relationship  Between  Dependent  Variables 

Intercorrelations  were  run  between  the  dependent  variables  to  determine  the  relationship  between 
achievement,  instructional  efficiency,  and  process  efficiency.  The  Pcarson-Product  Moment  correlation  coefficient 
of  combined  achievement  scores  and  instructional  efficiency,  r = 0.815,  shows  a strong,  direct  relationship  between 
the  two  variables.  The  Pearson  Product  Moment  correlation  coefficients  between  combined  achievement  and 
process  efficiency,  and  between  instructional  efficiency  and  process  efficiency,  r = 0.599  and  r = 0.63,  respectively, 
show  a moderate,  direct  relationship. 

Scatter  plots  of  the  dependent  variables  were  generated  to  visually  inspect  the  relationship  between 
dependent  variables  to  ascertain  any  nonlinear  or  curvilinear  relationships.  The  relationship  between 
achievement  and  instructional  efficiency  was  strong,  linear,  and  positive.  This  is  to  be  expected  as 
instructional  efficiency  is  the  dividend  of  achievement  divided  by  lime  spent  on  treatment.  The  relationship 
between  achievement  and  process  efficiency  was  moderate,  positive,  and  somewhat  linear.  The  relationship 
between  instructional  efficiency  and  process  efficiency  as  moderate,  positive,  and  again,  somewhat  linear. 

Results  Regarding  Research  Questions 

Multivariate  and  univariate  analyses  of  variance  were  used  to  analyze  data.  The  multivariate  analysis  of 
variance  addresses  all  main  effects  and  interaction  effects  of  the  instructional  strategics  examined  in  this  study  on 
the  dependent  variables  as  a whole.  In  contrast,  the  univariate  analyses  of  variance  address  the  main  effects  and 
interaction  effects  of  the  instructional  strategics  examined  in  this  study  on  each  dependent  measure  individually. 

Multivariate  Analysis  of  Variance 

As  the  three  dependent  variables  are  correlated  and  likely  share  a common  conceptual  meaning,  a three- 
way  multivariate  analysis  of  variance  was  used  to  analyze  the  data.  The  principle  advantage  of  such  a multivariate 
procedure  over  the  traditional  univariate  h tests  is  that  it  permits  a test  of  the  possible  interactions  among  multiple 
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dependent  measures  that  cannot  be  evaluated  if  each  dependent  measure  is  tested  in  isolation  (Huek,  Cormier,  & 
Bounds,  1974). 

The  Wilk’s  Lambda  criterion  for  multivariate  analysis  showed  there  is  no  significant  multivariate  effect 
attributable  to  organizational  strategy  {F  (3,  95)  = 0.765,  ps  <0.52),  delivery  strategy  (F  (3,  95)  = 0.2.27 1 ps  <0.09), 
or  management  strategy  (F  (3,  95)  = 0.940,  ps  <0.43).  The  multivariate  effect  for  the  interaction  of  organizational 
strategy  and  delivery  strategy,  F (3,  95)  ~ 2.348,  is  not  significant  at  ps  < 0.08.  In  addition,  the  multivariate 
difference  for  the  interaction  between  organizational  strategy  and  management  strategy,  and  delivery  strategy  and 
management  strategy  can  be  accounted  for  by  chance  alone  (Fs  - 1.32  and  0.43,  ps  < 21  and  .73,  respectively). 
Finally,  the  multivariate  effect  attributable  to  the  interaction  of  all  three  factors,  organizational  strategy,  delivery 
strategy,  and  management  strategy,  F (3,  95)  = 1.619  is  not  significant  at  ps  < 0. 19.  A summary  of  the  multivariate  F 
ratios  appears  in  Table  3. 


Table  3.  Multivariate  Analysis  of  Variance  (MAN  OVA)  of  Achievement,  Instructional  Efficiency,  and  Process 
Efficiency  by  Organizational  Strategy,  Delivery  Strategy,  and  Management  Strategy 


Source  of  Variance 

cr 

F ratio 

Timor 

F 

O rganlzational  Strategy  (OS) 

3/95 

0.976 

0.766 

0517 

D dl  vary  Strategy  (D  S) 

3/95 

0.933 

2271 

0085 

Management  Strategy  (MS) 

3/95 

0.971 

0.940 

0.425 

OSxDS 

3/95 

0.93 1 

2348 

0078 

OSxMS 

3/95 

0.960 

1.315 

0274 

DSxMS 

3/95 

0.987 

0.427 

0734 

OSxDSxMS 

3/95 

0.951 

1.619 

0.190 

Univariate  Analysis  of  Variance 

Although  the  multivariate  analysis  of  variar  ? for  all  three  dependent  measures  showed  no  significant  main 
or  interaction  effects  generated  by  the  instruction  . strategies  studied,  the  strong  but  not  significant  effect  of 
delivery  strategy  on  the  dependent  measures  {F  (3,95)  = 2.27,  ps  <09)  as  well  as  the  similarly  strong  but  not 
significant  interaction  effects  between  organizational  strategy  and  delivery  strategy  (F  (3,  95)  = 2.35,  ps  <0.08)  and 
the  moderate  interaction  effect  between  organizational  strategy,  delivery  strategy,  and  management  strategy  (F  (3. 

95)  = 1.619,  ps<020)  merits  additional  examination.  Means  and  standard  deviations  for  dependent  measures  for  all 
treatment  groups  arc  shown  in  Table  4. 

Achievement 

The  univariate  main  effect  of  organizational  strategy  on  achievement,  F = 1.77,  df-  1/97,  although  strong,  was  not 
significant  at  p < 0.19.  Varying  instructional  organization  strategies  (criterion  referenced  instruction  and  problem  based 
learning)  showed  no  significant  effect  on  achievement  when  learning  from  .cchnology  based  visual  literacy  instruction. 
Although  not  significant,  examination  of  die  cell  means  in  Table  4 shows  that  for  all  groups,  those  groups  receiving  the 
constructivist  problem-based  treatment  consistently  scored  higher  on  combined  achievement  than  did  those  groups  receiving 
the  objcctivist,  criterion-referenced  treatment.  The  relationship  between  organizational  strategy  and  achievement  is  shown  in 

The  univariate  main  effect  of  delivery  strategy  on  achievement,  F = 0.07,  df  = 1/97,  is  not  significant  at  p < 0.79. 
Varying  instructional  delivery  strategics  (local  computer-based  instruction  (CBI)  and  networked  hypermedia  based  (WWW) 
instruction  (NHBI))  shows  no  significant  effect  on  achievement  when  learning  from  technology  based  visual  literacy 
instruction.  Although  not  significant,  examination  of  the  cell  means  in  Table  4 shows  that  for  all  groups,  those  groups 
receiving  the  networked  hypermedia  based  instruction  consistently  scored  higher  on  combined  achievement  than  did  those 
groups  receiving  the  local  computer-based  treatment. 

The  univariate  main  effect  of  management  strategy  on  achievement,  F =0.001.  df  = 1/97,  is  not  significant  al  />  < 
0.97.  Varying  instructional  management  strategics  (individual  and  cooperative  task  structures)  shows  no  significant  effect  on 
achievement  when  learning  from  technology  based  visual  literacy  instruction.  Although  the  effect  of  management  strategy  on 
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instructional  efficiency  is  not  significant,  examination  of  the  cell  means  in  Table  4 shows  that  for  all  groups,  the  individual 
task  structure  groups  consistently  showed  a higher  achievement  score  than  did  the  cooperative  task,  structure  groups. 

Table  4 . Cell  Means  and  Standard  Deviations  Comparing  Organizational  Strategy,  Delivery  Strategy,  and 
Management  Strategy  for  Combined  Achievement,  Instructional  Efficiency,  and  Process  Efficiency 
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Dependent  Variable:  Process  Efficiency 
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Figure  2 . Combined  Achievement  Means  by  Organizational 
Strategy’  and  Delivery  Strategy 


The  univariate  interaction  effect  of  organizational  strategy  and  delivery  strategy,  organizational  strategy 
and  management  strategy,  and  delivery  strategy  and  management  strategy,  show  no  significant  effect  on 
achievement.  In  addition,  the  interaction  effect  of  organizational  strategy,  delivery  strategy,  and  management 
strategy  is  also  not  significant  (F  = 1.6,  df-  1/97,  p < 0.21).  The  instructional  strategies  studied  do  not  significantly 
interact  during  instruction  to  effect  achievement  when  learning  from  technology  based  visual  literacy  instruction. 

Instructional  Efficiency 

No  significant  main  or  interaction  effects  of  organizational  strategy,  delivery  strategy,  or  management 
strategy  on  instructional  efficiency  were  indicated  by  the  statistical  analysis.  Although  the  effects  of  organizational 
strategy,  delivery  strategy,  and  management  strategy  on  instructional  efficiency  are  not  significant  (F  = .1.67,  df  = 
1/97,  p < 0.2;  F = 2.36,  df = 1/97,  p < 0. 13,  and  F = .617  df=  1/97 , p < 0.43,  respectively),  examination  of  the  cell 
means  in  Table  4 shows  trends  similar  to  those  observed  with  the  dependent  measure,  achievement.  For  all  groups, 
those  groups  receiving  the  problem-based  treatment  consistently  showed  a higher  level  of  instructional  efficiency 
than  did  those  groups  receiving  the  objcctivist  treatment.  In  a similar  vein,  those  groups  receiving  the  networked 
hypermedia-based  treatment  consistently  showed  a higher  level  of  instructional  efficiency  than  did  those  groups 
receiving  the  local  computer-based  treatment.  Finally,  the  individual  task  structure  groups  consistently  showed  a 
higher  level  of  instructional  efficiency  than  did  the  cooperative  task  structure  groups. 

Process  Efficiency  Results 

No  significant  main  or  interaction  effects  of  organizational  strategy,  delivery  strategy,  or  management 
strategy  on  process  efficiency  were  indicated  by  the  statistical  analysis.  Although  the  effect  of  organizational 
strategy  on  process  efficiency  is  not  significant  {F  - 1.77,  df  - 1/97,  p < u.19),  examination  of  the  cell  means  in 
Tabic  4 shows  that  for  all  groups,  those  groups  receiving  the  objcctivist  CBI  and  objcctivist  NHBl  treatments 
performed  similarly.  In  comparison,  the  mean  process  efficiency  of  those  groups  receiving  the  problem-based  CBI 
decreased  slightly  from  their  objcctivist  counterparts.  The  mean  process  efficiency  of  the  problem-based  NHBl 
group  increased  strongly,  indicating  a probable  drop  in  the  number  of  notes  taken  with  no  corresponding  drop  in 
achievement  score  or  time  on  treatment.  The  relationship  between  organizational  strategy  and  instructional 
efficiency  is  shown  in  Figure  3. 
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Figure  3 . Process  Efficiency  Means  by  Organizational 
Strategy * and  Delivery  Strategy 


The  effect  of  delivery  strategy  on  process  efficiency  is  not  significant  ( F = 3.22,  df  - 1/97,  p < 0.08).  As 
this  is  the  strongest  F ratio  of  the  entire  study,  this  relationship  bears  some  further  examination.  Examination  of  the 
ceil  means  in  Table  15  shows  that  for  all  groups,  all  groups  receiving  the  local  computer-based  treatment  and  the 
groups  receiving  the  objcctivist  networked  hypermedia-based  treatment  consistently  showed  similar  levels  of 
process  efficiency.  Those  groups  receiving  the  problem-based  networked  hypermedia-based  treatments  showed 
considerably  higher  levels  of  process  efficiency. 

The  effect  of  management  strategy  on  process  efficiency  is  also  not  significant  (F  = .1.37  df  = 1/97,  p < 
0..24).  However,  examination  of  the  cell  means  in  Tabic  4 shows  that  for  all  groups,  the  individual  task  structure 
groups  consistently  showed  a higher  level  of  process  efficiency  than  did  the  cooperative  task  structure  groups. 

Discussion 

Discussion  of  Research  Findings  by  Organizational  Strategy 

MANOVA  and  ANOVA  results  indicated  that  there  were  no  statistically  significant  differences  in  the 
amount  of  achievement,  instructional  efficiency,  or  processing  efficiency  associated  with  instructional  organization 
strategies  (criterion  referenced  instruction  and  problem  based  learning)  when  learning  from  technology  based  visual 
literacy  instruction.  Interesting  trends,  however,  were  observed  in  the  data  associated  with  organizational  strategy. 
Groups  receiving  the  constructivist  problem-based  treatment  consistently  scored  higher  on  combined  achievement 
that  did  those  groups  receiving  the  objcctivist,  criterion-referenced  treatment.  The  corresponding  F statistic 
associated  with  the  univariate  main  effect  of  organizational  strategy  on  achievement,  F = 1.77,  df  = 1/97,  although 
no*  significant,  is  strong  at  p < 0. 1 9. 

Those  groups  receiving  the  problem-based  treatment  also  consistently  showed  a higher  level  of 
instructional  efficiency  that  did  those  groups  receiving  the  objcctivist  treatment.  This  was  not  expected,  as  the 
problem-based  treatment  gave  learners  no  recommended  path  to  review  content.  However,  they  showed  no  increase 
in  time  which  would  have  decreased  their  instructional  efficiency. 

A similar  type  of  relationship  is  seen  between  organizational  strategy  and  process  efficiency.  Groups 
receiving  the  objcctivist  CBI  and  objcctivist  NHBl  treatments  performed  similarly.  In  comparison,  the  mean  process 
efficiency  of  those  groups  receiving  the  problem-based  CBI  decreased  slightly  from  their  objcctivist  counterparts. 


The  mean  process  efficiency  of  ihc  problem-based  NHBI  group  increased  strongly,  indicating  a probable  drop  in  the 
number  of  notes  taken  with  no  corresponding  drop  in  achievement  score  or  time  on  treatment. 

The  design  of  treatments  employing  different  organizational  strategies  was  tightly  controlled  to  maintain 
equivalent  content  and  content  organization.  In  other  words,  although  some  effort  was  made  to  make  the  objectivist, 
criterion-referenced  treatment  and  the  constructivist,  problem-based  treatment  dissimilar,  a strong  effort  was  also 
made  to  ensure  that  any  effect  seen  would  not  be  due  to  a difference  in  the  amount  or  nature  of  content  received,  a 
problem  noted  by  Miclkc  (1968)  in  early  media-based  research.  If  these  restrictions  were  loosened  somewhat, 
thereby  increasingly  polarizing  the  treatments,  more  significant  differences  in  the  achievement,  instructional 
efficiency,  and  process  efficiency  levels  of  the  treatment  groups  might  be  observed. 

Discussion  of  Research  Findings  by  Delivery  Strategy 

MANOVA  and  ANOVA  results  indicated  that  there  were  no  statistically  significant  differences  in  the 
amount  of  achievement,  instructional  efficiency,  or  processing  efficiency  associated  with  varying  instructional 
delivery  strategies  (local  computer-based  instruction  and  networked  hypermedia-based  instruction)  when  learning 
from  technology-based  visual  literacy  instruction.  It  should  be  noted  that  of  all  the  instructional  strategies  examined, 
delivery  strategies  showed  the  strongest  effect  on  the  dependent  measures.  The  multivariate  analysis  of  variance 
showed  the  main  effect  generated  by  delivery  strategies  on  all  dependent  variables  combined  was  strong,  but  not 
significant  at  F (3, 95)  = 2.27,  ps  <0.09. 

The  univariate  main  effect  generated  by  delivery  strategy  on  achievement  was  not  strong  at  F =0.07,  ell  = 
1/97,  p < 0.79.  Groups  receiving  the  networked  hypermedia-based  treatment  consistently  showed  a higher  combined 
achievement  score  and  showed  a higher  level  of  instructional  efficiency  than  did  those  groups  receiving  the  local 
computer-based  treatment.  This  was  not  expected  as  the  researcher  assumed  the  networked  hypermedia-based 
treatment  would  require  greater  time  to  access  and  download  individual  pages.  In  retrospect,  the  learners  did  not 
have  to  wait  to  move  from  one  screen  of  chunked  content  to  the  next  as  they  did  with  the  local  computer-based 
treatment,  so  this  likely  accounted  for  any  decrease  in  time  spent  on  the  treatment,  thus  increasing  instructional 
efficiency. 

A similar  type  of  relationship  is  seen  between  delivery  strategy  and  process  efficiency.  Groups  receiving 
the  local  computer-based  treatment  and  the  groups  receiving  the  objectivist  networked  hypermedia-based  treatment 
showed  similar  levels  of  process  efficiency.  Those  groups  receiving  the  problem-based  networked  hypermedia- 
based  treatments  showed  considerably  higher  levels  of  process  efficiency,  indicating  a possible  interaction  effect 
between  organizational  strategy  and  delivery  strategy  on  process  efficiency. 

Although  there  arc  an  increasing  number  of  WWW-page  design  guides  (see,  for  example,  Apple  Web 
Design  Guide,  1996;  Schwicr  & Misanchuk,  1996,  February),  little  has  been  written  giving  recommendations  or 
guidelines  for  creating  instructionally  efficient  WWW  pages.  Indeed,  much  of  educational  use  of  the  Internet  has 
been  archival  in  nature  (lecture  notes,  slide  shows,  and  visual  representations)  (Shotsberger,  199/r  . With  no 
recommended  instructional  screen  design  guidelines,  the  design  team  sought  to  create  instructionally  effective  Web 
pages,  extrapolating  several  screen  design  guidelines  from  computer  based  instruction  and  balancing  them  with 
measures  to  decrease  download  time  of  pages  and  images  within  pages.  Hannafin  (1985)  proposes  that  the  more 
rapidly  program  segments  arc  accessed,  the  more  effective  the  instruction.  In  order  to  keep  the  level  and  nature  of 
content  presentation  similar  across  treatments,  it  is  likely  that  the  design  team  did  not  take  full  advantage  of  the  node 
and  link  structure  of  the  Internet.  This  minimal  levels  of  links  available  most  likely  also  decreased  the  level  of 
disorientation  felt  by  subjects,  as  is  often  the  ease  in  Web-based  learning  (Tripp  & Roby,  1990).  In  addition,  visual 
images  were  presented  in  a smaller  format  than  they  were  on  the  local  CBI  treatments  in  order  to  decrease  download 
time.  As  was  the  case  with  organizational  strategies,  increased  polarization  of  the  delivery  strategy  treatments  might 
result  in  more  significant  differences  in  the  achievement,  instructional  efficiency,  and  process  efficiency  levels  of 
the  treatment  groups. 


Discussion  of  Research  Findings  by  Management  Strategy 

MANOVA  and  ANOVA  results  indicated  that  there  were  no  statistically  significant  differences  in  the 
amount  of  achievement,  instructional  efficiency,  or  processing  efficiency  associated  with  varying  instructional 
management  strategics  (individual  and  cooperative  task  structures)  when  learning  from  technology 'based  visual 
literacy  instruction.  Groups  receiving  instruction  in  an  individual  task  structure  consistently  showed  only  a slightly 
higher  combined  achievement  score  than  did  those  groups  receiving  instruction  in  cooperative  dyads.  This  supports 
earlier  research  by  Carrier  and  Sales  (1987)  and  Shlcchtcr  (1990)  that  found  students  who  complete  CBI  in 
cooperative  groupings  generally  perform  as  well  as  students  who  work  alone. 

In  addition,  those  groups  receiving  instruction  in  individual  task  structures  consistently  showed  a higher 
level  of  instructional  efficiency  that  did  those  groups  receiving  instruction  in  cooperative  dyads  This  is  to  be 
expected,  as  an  interaction,  meaningful  or  otherwise,  would  cause  the  cooperative  groupings  to  spend  more  time  on 
the  instructional  treatment,  thereby  decreasing  instructional  efficiency.  Instructional  efficiency  was  considerably 
lower  for  those  groups  receiving  the  objectivist  instructional  treatment  in  a cooperative  task  structure. 

A similar  type  of  relationship  is  seen  between  management  strategy  and  process  efficiency.  Groups 
receiving  instruction  in  individual  task  structures  once  again  showed  a higher  level  of  process  efficiency  than  those 
groups  receiving  instruction  in  cooperative  dyads.  Since  the  instructional  efficiency  was  higher  for  groups  receiving 
instruction  individually,  indicating  they  spent  less  time  on  the  treatment,  a higher  level  of  process  efficiency  for  the 
individual  groups  indicates  that  the  individual  groups  look  less  notes  than  the  cooperative  dyads,  resulting  in  a 
higher  level  of  process  efficiency. 

Discussion  of  Research  Findings  by  Interactions  of  Instructional  Strategies 

Of  all  the  interaction  combinations  of  instructional  strategies  possible,  only  organizational  strategy  and 
delivery  strategy  showed  any  potential  of  an  interaction  effect  on  any  dependent  variable. 

The  multivariate  analysis  of  variance  showed  the  interaction  effect  generated  by  organizational  strategy  and 
delivery  strategy  on  all  dependent  variables  combined  was  nearly  significant  at  F (3,  95)  = 2.34,  ps  <0.08. 
Univariate  analysis  of  variance  showed  the  interaction  effect  generated  by  organizational  strategy  and  delivery 
strategy  on  achievement  to  be  not  significant  (F  = 0.15,  d/=  1 , 97.  p <0.7).  Univariate  analysis  of  variance  showed 
the  interaction  effect  generated  by  organizational  strategy  and  delivery  strategy  on  instructional  efficiency  also  to  be 
not  significant  (F  = 0.07,  df-  1 , 97.  p <0.79). 

In  contrast,  univariate  analysis  of  variance  showed  the  interaction  effect  generated  by  organizational 
strategy  and  delivery  strategy  on  process  efficiency  to  be  nearly  significant  (F  = 3.26  df  ~ ),  97 .p  <0.7).  Examining 
Figure  3,  a visual  display  of  process  efficiency  means  by  delivery  strategy  and  organizational  strategy,  the 
interaction  direction  can  be  seen. 

The  relationship  can  be  described  as  follows.  Groups  which  received  the  objectivist  treatment  showed 
relatively  the  same  level  of  process  efficiency  regardless  of  delivery  strategy.  Groups  which  received  the  objectivist, 
networked  hypcrmcdia-bascd  treatment  also  demonstrated  approximately  the  same  level  of  process  efficiency.  The 
groups  which  received  the  problem-based,  networked  hypermedia  based  treatments,  however,  displayed 
substantially  higher  levels  of  process  efficiency.  Since  instructional  efficiency  does  not  show  a similar  potential 
interaction  effect,  it  can  be  surmised  that  the  potential  interaction  effect  is  likely  due  to  a decrease  in  the  quantity  of 
notes  taken  by  these  groups,  with  no  accompanying  decrease  in  achievement  scores  or  change  in  the  amount  of  time 
spent  on  treatment. 

The  reasoning  for  this  potential  interaction  effect  can  only  be  hypothesized.  One  reason  may  be  that  as 
proposed  by  Borsook  and  Higginbolham-Wheat  (1992,  February),  the  node  and  link  structure  of  networked 
hypermedia-based  instruction  may  in  some  way  mirror  the  linked  structure  of  mental  schema.  Kcarslcy  (1988)  offers 
that  hypertext  or  hypermedia  systems  like  the  World  Wide  Web,  improve  learning  by  focusing  attention  on  the 
relationships  between  ideas  rather  than  on  isolated  facts.  The  associations  provided  by  the  links  in  NHBI,  Kearsley 
offers,  should  facilitate  retrieval,  concept  formation,  and  comprehension.  If  this  is  the  case,  further  exploration  of  the 
application  of  constructivist  learning  theory  to  the  design  of  Web-based  instruction  is  certainly  called  for. 


Implications  and  Recommendations 

These  results  indicate  that  instructional  strategics  may  be  an  important  variable  in  learning  from 
technology-based  instruction.  However,  further  research  is  needed  to  determine  which  aspects  of  each  type  of 
instructional  strategy  effects  learning  in  emerging  technology-based  instruction.  In  particular,  the  treatments  used  in 
this  study  need  to  be  refined  in  order  to  increase  the  differences  between  factor  levels,  thus  increasing  the  power  of 
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the  test.  In  addition,  additional  variables  need  to  be  researched,  including  aptitude,  sclf-directedness,  and  motivation. 

Although  this  study  offered  no  statistically  significant  results,  it  docs  open  the  door  for  many  research 
paths.  At  the  very  least,  the  lack  of  effect  between  treatments  indicate  that  alternative  methods  and  technologies  of 
teaching  and  learning,  including  constructivist  learning,  networked  hypermedia-based  instruction,  and  cooperative 
learning  deserve  additional  study. 
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Abstract 

The  design  and  development  of  Computer-Based  Instruction  (CBI)  is  a complex  process.  This  paper 
describes  how  one  Wayne  State  University  multimedia  design  team  is  applying  Keller’s  ARCS  Model  to  the  entire 
process  of  design,  development,  and  evaluation  of  multimedia  courseware.  The  ARCS  Model  has  been  applied  to 
the  prototype  module  and  is  being  incorporated  into  all  present  and  future  modular  designs.  The  design  team  is 
working  with  faculty  members  from  five  universities  to  design  and  develop  multimedia  courseware  for  college 
engineering  courses  to  be  delivered  at  a National  Science  Foundation  project  located  at  a manufacturing  facility.  A 
brief  summary  of  Keller’s  ARCS  Model  is  presented.  The  primary  emphasis  of  the  article  is  the  description  of  the 
CBI  design  process  and  resulting  product  design  and  features  as  they  relate  to  the  ARCS  Model.  Issues  that  continue 
to  challenge  the  design  team  arc  presented  in  the  conclusion. 

Establishing  an  Innovative  Engineering  Education  Program 

Just  as  a well-designed  building  depends  on  having  a solid  foundation,  so  docs  an  instructional  product. 
Engineering  education  emphasizes  technology  and  technological  innovations.  Multimedia  Computer-Based 
Instruction  (CBI)  is  a dynamic,  interactive  technology.  Motivating  students  with  relevancy  in  instruction  increases 
their  learning  of  theoretical  concepts.  Combining  these  three  elements-engineering  education,  multimedia 
instruction,  and  relevancy  in  instruction-can  have  a significant  impact  on  the  way  students  learn  to  become 
engineers. 

These  simple  but  profound  statements  have  driven  the  Greenfield  Coalition  for  New  Manufacturing 
Education  in  the  establishment  of  a new  paradigm  for  engineering  education.  Furthermore,  it  is  the  unique 
combination  of  these  factors  that  was  the  primary  impetus  for  using  the  ARCS  Model  to  design  multimedia  college 
engineering  courses. 

Funded  by  the  National  Science  Foundation  in  1993,  the  Greenfield  Coalition  for  New  Manufacturing 
Education  is  comprised  of  five  universities,  six  industry  partners,  and  a private  agency.  The  original  commitment 
was  to  establish  a virtual  university  at  Detroit’s  Focus:HOPE’s  Center  for  Advanced  Technologies  (CAT),  a private 
agency  manufacturing  facility  originally  funded  primarily  by  the  U.S.  Department  of  Defense.  The  university 
partners  (Wayne  State  University,  University  of  Michigan,  University  of  Detroit  Mercy,  Lawrence  Technological 
University,  and  Lehigh  University)  are  committed  to  providing  top-quality  engineering  curricula;  the  industry 
partners  (GM,  Ford,  EDS.  Chrysler,  Cincinnati  Milacron,  and  Detroit  Diesel)  arc  committed  to  providing  equipment, 
contracts,  and  advisors;  and  Focus:HOPE  is  committed  to  providing  the  manufacturing  facility,  the  classrooms,  and 
the  candidates  for  the  degree  program.  The  150  full-time  workers  in  the  facility  arc  the  students  in  the  “virtual 
university.”  The  delivery  of  credit-bearing  engineering  courses  is  being  accomplished  primarily  through  “Innovative 
Instructional  Processes”  (IIPs).  Consequently,  from  the  outset  the  coalition  adopted  the  following  goals: 

1 ) Create  accredited  degree  programs--an  associate’s  and  bachelor’s  degrees; 

2)  Ensure  that  students  will  obtain  the  planned  degrees  primarily  via  the  coalition’s  instructional  modules; 

3)  Change  from  a teacher-student  system  to  a coach-learner  approach; 

4)  Develop  a curriculum  that  will  be  used  not  only  by  the  coalition’s  college  but  also  by  the  participating 
universities;  and 

5)  Utilize  CBI  modules  as  an  integral  part  of  the  educational  foundation. 

In  addition,  the  faculty  developers  were  challenged  to  incorporate  manufacturing  relevance  into  the  modules. 
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In  the  first  three  years,  the  faculty  members  (25-30  developers)  from  the  partner  universities  were  awarded 
contracts  and  were  asked  to  design  and  develop  their  proposed  CBI  modules  using  Authorwarc  Professional©,  an 
icon-based  authoring  program.  Although  most  faculty  developers  were  experts  in  their  engineering  fields,  they  were 
not  instructional  designers  or  multimedia  programmers.  Unfortunately,  the  faculty  developers  were  not  given 
consistent  guidelines,  standards,  templates,  or  much  of  anything  in  the  way  of  professional  support  for  the  design 
and  development  of  the  CBI.  Some  university  faculty  members  made  impressive  efforts  in  their  design  and 
development  projects.  Others  floundered,  however.  Consequently,  the  various  modules  were  often  little  more  than 
text-filled  page-turners  with  sparse  media,  confusing  navigation,  and  limited  learner  control  or  interaction.  The 
faculty  developers  delivered  these  CBI  modules  along  with  their  classroom  instruction  to  the  candidates  at  the  CAT 
with  varied  levels  of  success, 

Establishing  a CBI  Team  to  Design  and  Develop  Multimedia  Courseware 

In  October  1996,  NSF  reviewed  the  CBI  modules  and  concluded  that  the  Greenfield  Coalition  should 
centralize  production  to  improve  the  “quality  and  quantity”  of  the  modules.  This  meant  that  the  entire  design  and 
development  process  would  undergo  significant  changes.  To  ■xccomplish  this  major  shift,  the  coalition  established  a 
professional  multimedia  design  team  comprised  of  an  instructional  designer,  programmer,  and  a media  specialist. 
Within  six  months  the  team  added  a graphics  designer,  writcr/cditor,  multimedia  specialist,  and  second 
programmer. 

A CBI  structure,  or  a basic  blueprint,  was  established  as  the  template  for  lesson  presentation.  All  elements 
of  a lesson  center  on  the  core  material,  which  may  be  a text,  series  of  articles,  CD-ROM  or  other  existing  body  of 
knowledge.  In  essence,  then,  the  new  CBI  modules  would  not  attempt  to  contain  the  entire  course,  but  the  CBI 
modules  would  be  connected  to  a designated  core  material  and  contain  only  key  elements  that  are  best  delivered 
through  multimedia.  In  this  new  structure,  a basic  module  is  defined  as  having  three  to  five  topics  with  matching 
learning  objectives  with  50  to  100  screens  (including  3-5  screens  for  a self-chcck  and  5-15  screens  for  a module 
assessment  that  has  a test  bank  of  questions).  The  student  should  be  able  to  complete  the  lesson  in  30  to  90  minutes. 
Each  module  contains  an  introduction,  stated  learning  objectives,  summary  of  the  core  material  lesson,  illustrations 
and  examples,  simulations  or  “what  if’  questions,  self  assessment  and  a module  assessment.  In  addition,  modules 
may  be  linked  to  external  software  programs. 

The  learner’s  activities  are  tracked  by  a Computer-Managed  Instruction  (CMI)  system.  The  student  logs  on 
to  the  client-server  system,  selects  a course  from  the  curriculum,  selects  a module,  then  selects  activities  within  the 
module.  The  tracking  of  the  learner’s  log  on  and  selection  is  accomplished  through  Solis  Pathware©,  a product  of 
Macromedia. 

Committing  to  an  ISD  Approach  and  a Fully-Featured  Prototype 

To  ensure  that  the  design  process  and  the  CBI  structure  would  be  implemented  effectively,  the  coalition 
team  made  a commitment  to  (1)  an  ISD  approach  and  (2)  a fully  featured  CBI  prototype. 

• Commitment  to  an  ISD  approach.  The  team  decided  early  on  that  established  Instructional  Systems  Design 
models  and  tenets  would  be  the  foundation  for  design  and  development.  The  Dick  and  Carey  Model  (1996)  was 
applied  to  the  macro  design  of  the  entire  CBI  process.  Keller’s  ARCS  Model  (Kcllerl987a;  1987b;  1987c; 
Keller  & Kopp,  1987;  Keller  & Suzuki,  1988)  was  applied  to  the  design  of  the  program  interface,  screen  layout, 
and  the  CBI  lesson  presentation  and  on-line  assessment.  Furthermore,  the  ARCS  Model  was  applied  to  the 
product  evaluation  process. 

• Commitment  to  a fully  featured  C31  prototype  based  on  our  CBI  structure.  This  meant  that  the  team  would 
design  a CBI  structure  that  would  fit  all  the  knowledge  areas  in  the  engineering  curriculum.  Each  module  would 
have  consistent  screen  design  (menu  bar,  mapping  area,  font  type  and  size,  text  and  media  placement);  standard 
navigation  with  alternative  navigation  methods  to  emphasize  user  control;  on-line  access  to  information; 
appropriate  use  of  multimedia  (graphics,  animation,  video,  sound,  simulations);  and  ample  learner  interactions. 
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ARCS:  The  Motivational  Theory  and  Model 

A core  tenet  of  traditional  instructional  design  theory  and  practice  is  that  design  should  be  a systematic 
process  (Gagne,  1985;  Gagne,  Briggs  & Wager,  1988;  Rothwell  & Kazanas,  1992;  Seels  & Richey,  1994).  Because 
of  the  acceptance  of  a systematic  approach  designers  generally  follow  an  ISD  model,  which  not  only  describes  and 
explains  the  design  elements  but  also  prescribes  the  required  activities  to  achieve  the  desired  objectives.  Educators 
generally  consider  motivation  a relevant  component  in  both  learning  and  instruction.  In  his  article,  “Motivation  and 
Instructional  Design:  A Theoretical  Perspective,”  Keller  (1979)  presents  a theoretical  framework  for  the  inclusion  of 
motivation  in  the  theory  and  practice  of  instructional  systems  design.  Keller  (1979;  1983)  claims  that  if  motivation  is 
to  be  applied  effectively  to  instructional  design,  it  should  be  based  on  solid  motivation  theory  and  a systematic 
process  of  instructional  design.  One  of  the  key  ingredients  in  the  successful  implementation  of  the  newly  established 
Greenfield  Coalition  process  was  to  apply  the  ARCS  Model  to  the  design  of  multimedia  college  engineering 
courses. 

Keller’s  ARCS  Model  of  Motivational  Design  (Keller,  1987a;  1987b;  1987c)  works  with  a traditional  ISD 
model.(See  Table  1.)  The  acronym  ARCS  refers  to  the  four  key  elements  of  motivation:  A,  Attention;  R,  Relevance; 
C,  Confidence;  and  S,  Satisfaction. 

Table  1.  The  Interface  of  Motivational  and  Instructional  Design 


Phase 

Instructional  Design  Steps 

Motivational  Design  Steps 

Define 

Pre-project  analysis-project 
Conduct  task,  job,  or  content  analysis 
Conduct  instructional  analysis 
Identify  audience  entry  behaviors 
Write  performance  objectives  and 
criterion  measures 

Conduct  audience  motivational  analysis 

Write  motivational  objectives  and  criterion 

measures 

Design 

Design  instructional  sequences 
Instructional  methods 

Generate  motivational  strategies 
Select  strategies 

Integrate  motivational  and  instructional  strategics 

Develop 

Select  or  create  instructional  materials 

Develop  test  for  learning  and 
performance 

Prepare  motivational  materials 
Enhance  instructional  materials 
Develop  test  for  motivation 

Pilot 

_gaph97 

Implement  with  test  population  representatives 
Conduct  formative  evaluation 
Certify  or  revise 

Note.  From  “The  systematic  process  of  motivational  design’1  by  J.M.  Keller,  1987,  Performance  and  Instruction,  26, 
p.  7. 


Most  design  models  include  the  basic  steps  of  (1)  define,  (2)  design,  (3)  develop,  (4)  implement,  and  (5) 
evaluate.  The  ARCS  Model  combines  the  last  two  steps  of  implement  and  evaluation  into  one,  “pilot.”  Others  in  the 
field  support  Keller's  claim  that  the  motivational  model  fits  well  with  a traditional  ISD  model.  For  example,  Okcy 
and  Santiago  (1991)  demonstrate  the  model’s  value  by  using  it  with  Gagne’s  Events  of  Instruction  and  Dick  and 
Carey’s  design  model.  Main  (1993)  also  showed  how  the  motivation  model  could  be  used  effectively  with  a 
traditional  design  model.  Each  major  category  has  three  sub-categories  to  be  analyzed  for  the  development  of 
appropriate  motivational  strategies.  The  process  questions  guide  the  designer’s  focus  and  selection  of  strategics. 
(See  Table  2.) 
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Table  2.  Motivational  Categories  of  the  ARCS  Model 


Categories  & Sub-categories 

Process  Questions 

Attention 

A.  1. 

Perceptual  Arousal 

• 

What  can  I do  to  capture  their  interest? 

A. 2. 

Inquiry  Arousal 

• 

How  can  I stimulate  an  altitude  of  inquiry? 

A. 3. 

Variability 

• 

How  can  I maintain  their  attention? 

Relevance 

R.  1. 

Goal  Orientation 

• 

How  can  I best  meet  my  learner’s  needs? 

R.2. 

Motive  Matching 

• 

How  and  when  can  I provide  my  learners  with  appropriate 
choices,  responsibilities,  & influences? 

R.3. 

Familiarity 

• 

How  can  I tie  the  instruction  to  the  learner’s  experiences? 

Confidence 

C.l. 

Learning  Requirements 

• 

How  can  I assist  in  building  a positive  expectation  for 
success? 

C.2. 

Success  Opportunities 

• 

How  will  the  learning  experience  support  or  enhance  the 
students’  beliefs  in  their  competence? 

C.3. 

Personal  Control 

• 

How  will  the  learners  clearly  know  their  success  is  based  on 
their  efforts  and  abilities? 

Satisfaction 

S.l. 

Natural  Consequences 

• 

How  can  I provide  meaningful  opportunities  for  learners  to 
use  their  newly  acquired  knowledge/skill? 

S.2. 

Positive  Consequences 

What  will  provide  reinforcement  to  the  learner’s  successes? 

S.3. 

Equity 

• 

Hov  can  I assist  the  students  in  anchoring  a positive  feeling 
about  their  accomplishments? 

jNote.  From  “The  Systematic  Process  of  Design,  “ by  J.M.  Keller,  1987,  Performance  and  Instruction,  26.  p.  2. 


Applying  the  ARCS  Model  to  the  Design  of  Multimedia  Engineering  Courseware 

To  design  and  develop  the  CBI  prototype,  the  Greenfield  Coalition  CBI  design  team  worked  with  two 
professors  from  Wayne  State  University  on  a course  in  engineering  economics.  The  resulting  CBI  screen  design 
serves  as  a model  for  present  and  future  courseware  development,  (See  Figure  1.)  The  four  elements  of  the  ARCS 
Model,  adult  learning  theory,  and  multimedia  design  were  used  to  guide  design  decisions. 

Attention 

Attention,  as  Keller  points  out,  is  the  first  condition  of  motivation  and  a prerequisite  for  learning.  He  argues 
that  it  is  not  enough  to  gain  the  learner’s  attention,  the  “...real  challenge  is  to  sustain  it,  to  produce  a satisfactory 
level  of  attention  throughout  a period  of  instruction”  (p.  3).  Keller’s  (1987b)  definition  for  attention  is  “capturing  the 
interest  of  learners;  stimulating  the  curiosity  to  learn”  (p.  2). 

To  gain  and  sustain  the  learner’s  attention,  the  design  team  incorporated  multimedia  into  the  Introduction; 
provided  relevant  graphics,  video,  audio,  and  animation  throughout  the  lessons;  and  included  ample  learner 
interactions.  Attention  to  the  lesson  is  emphasized  with  the  use  of  consistent  placement  of  screen  title,  key  words, 
objectives,  summary,  appropriate  media  and  readable  text,  Although  it  may  seem  a simple  accomplishment, 
restricting  the  text  display  area  to  ten  or  less  lines  of  16  pt.  text  on  a screen  was  a major  hurdle  to  overcome.  Our 
experience  with  most  professors  is  that  they  want  to  put  as  much  text  as  possible  on  the  screen.  However,  to  catch 
and  sustain  the  learner’s  attention,  the  text  needs  catch  the  reader’s  eye,  be  easily  readable  and  understandable.  This 
generally  means  shorter  text  elements.  (See  Figure  1.) 
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Relevance 

Relevance,  Keller  (1987b)  emphasizes,  “is  a powerful  factor  in  determining  what  we  arc  motivated  to  learn, 
or  what  we  are  willing  to  continue  paying  attention  to  after  our  attention  has  been  aroused  or  rcstimulated”  (p.  3). 
Relevance  is  defined  as  “meeting  the  personal  nccds/goals  of  the  learner  to  effect  a positive  attitude'*  (p.  2).  In  a 
recent  study  with  100  college  students,  relevance  was  determined  to  be  the  most  important  motivational  strategy 
(Means  ct  al.,  1997).  The  study  investigated  the  comparative  efficacy  of  intrinsic  relevance  of  instruction  and 
embedded  relevance  strategies  as  recommend  by  the  ARCS  Model  of  instruction.  The  researchers  argue  that 
“relevance  strategics  increase  the  meaningfulness  of  instruction  by  relating  it  to  personal  needs”  (p.  7). 

To  increase  relevance  of  the  content  to  manufacturing,  the  CBI  team  designed  in  media  ''video,  audio, 
animation,  and  graphics)  that  illustrated  concepts  and  applications  for  the  adult  learner  in  the  context  of 
manufacturing.  The  faculty  developers  work  closely  with  industry  partners  and  representatives  from  Focuv.HOPE  to 
find  relevant  case  studies,  projects,  and  other  experiential  opportunities  for  incorporation  into  the  module  and  the 
entire  class.  These  elements  are  evident  in  the  screen  design  shown  in  Figure  1 . 

Figure  /,  Standard  Screen  Design  for  Multimedia  Development 


Confidence 

Confidence,  (Keller  & Kopp,  1987)  involves  the  learner’s  “positive  expectancy  for  success”  (p.  294).  There 
are  many  factors  that  have  an  influence  on  a learner’s  confidence  or  expectancy  for  success,  but  the  authors 
emphasize  the  complexity  of  the  confidence  factor  in  this  statement:  “Adults,  like  children,  who  are  expected  to 
learn  a new  skill  need  an  opportunity  to  acquire  and  practice  the  skill  under  conditions  where  the  psychological  risks 
arc  reasonably  low”  (p.  294). 

Providing  learners  the  opportunity  to  practice  and  acquire  new  skills  under  low-risk  conditions  was  a key 
component  of  the  design.  To  build  the  learner's  confidence,  the  design  team  concentrated  on  establishing  three  main 
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elements:  (1)  a consistent,  logical  interface  with  ample  user  control;  (2)  embedded  on-line  support  of  the  module; 
and  (3)  appropriate  interactions. 

At  the  top  of  the  basic  screen  layout  is  the  Menu  Bar  with  six  pull  down  menus;  File,  Documents , and 
Glossary,  Try • Go  To , and  Help.  The  menu  bar  was  selected  to  imitate  the  familiar  Windows  presentation  style. 

The  menu  options  arc  explained  below.  (See  Table  3.) 


Table  3.  Menu  Bar  Options 


File  Click  File  to  select  Quit , Find , or  Return  to  Main  Menu.  Quit  is  alternate  navigation  that  lakes 

the  user  to  the  Windows  desktop,  bypassing  the  Computer-Managed  Instruction  (CMI)  home 
screen.  Quit  is  also  available  in  the  Navigation  bar  at  the  bottom  of  the  screen.  Find  allows  the 
user  to  search  for  all  instances  of  a word,  phrase,  or  sentence.  Return  to  Main  Menu  allows  the 
user  to  return  to  the  Computer  Managed  Instruction  (CMI)  home  screen.  Return  to  Main  Menu 
is  also  available  with  the  Main  button  in  the  Navigation  bar  at  the  bottom  of  the  screen. 

Click  Documents  to  display  a list  of  all  electronic  Portable  Document  Files  (PDFs)  that  the 
Documents  student  can  print.  The  documents  are  PostScript  files  that  are  accessed  through  Adobe  Acrobat 
6.  Consistent  throughout  the  courses  are  the  following  document  types:  Module  Guide 
(outline  of  the  lesson),  list  of  key  words  and  definitions  for  the  module,  list  of  objectives  for 
both  Module  and  Topic,  and  summary.  Selected  documents  that  the  faculty  include  are; 
sample  in-class  test,  list  of  formulas,  and  similar  printable  documents  that  would  enhance  their 
confidence  in  obtaining  needed  resource  material  for  the  class. 

Click  Glossary  to  be  taken  to  an  application,  which  allows  the  user  to  look  up  all  words,  and 
Glossary  definitions  used  in  the  Engineering  Curriculum.  This  feature  ties  all  the  courses  together. 

Definitions  can  be  searched  by  the  word  or  phrase  or  by  the  beginning  letter  of  the  word. 

Try ’ It  Click  Try  It  for  alternate  navigation  to  select  from  the  topic’s  interactions,  practice  exercises, 

and  self-check  opportunities.  Students  must  select  Return  to  Main  Menu  to  access  Self-Cheek, 
so  the  Self-Cheek  will  not  be  encountered  as  a student  progresses  through  the  lesson.  These 
can  also  be  accessed  as  the  student  progresses  through  the  lesson.  A Factor  Calculator  that 
allows  the  student  to  change  variables  for  the  formulas  and  view  the  changes  in  the  cash  How 
diagram  is  available  in  the  Engineering  Economies  modules. 

Go  To  Click  Go  To  for  a selection  of  all  screen  numbers  and  titles.  This  is  alternate  navigation  that 

allows  the  user  to  highlight  the  screen  he  or  she  wants  and  go  directly  to  it. 

Help  Click  Help  to  display  these  options  regarding  the  program:  Program  Tutorial , Quick  Help,  and 

Navigation  Methods. 


The  Menu  Bar  options  arc  designed  to  increase  the  learner’s  access  to  embedded,  on-line  content 
information,  alternate  navigation  choices,  and  guidance  about  the  program’s  interface. 

The  left  side  of  the  screen  is  the  mapping  area,  which  contains  advance  organizers.  Advance  organizers 
alert  the  learner  to  key  aspects  of  the  lesson  (Ausubcl,  1968).  At  the  top,  directly  under  the  File  on  the  menu  bar  is 
the  Topic  Title  of  Module.  This  is  displayed  on  each  screen  to  focus  the  learner’s  attention  on  the  section  of  the 
module  he  or  she  is  in.  Under  the  title  arc  two  hotspots:  Objectives  and  Summary.  A click  on  Objectives  brings  up  a 
pop-up  window  of  the  topic’s  objectives.  A click  on  Summary'  brings  up  the  topic’s  last  screen,  which  contains  topic 
summary  information.  The  Key  Words  list,  which  is  in  a box  with  a scroll  bar,  contains  all  the  hypertext-words  in  the 
module.  Highlight  and  click  on  a selected  word  to  bring  up  the  definition  from  a database.  (These  words  arc  also 
available  in  the  Glossary  and  the  Key  Words  list  in  the  Documents  Pulldown  Menu.) 

Below  the  Key  Words  list  are  the  Tools  with  buttons  for:  Notetaker,  Audio,  Quick  Help,  and  a Calculator. 
Click  on  the  Quick  Help  to  access  an  interactive  help  screen  of  the  screen  layout  areas  and  navigation  buttons.  Click 
on  Notetaker  to  access  a window  that  allows  the  learner  to  take  notes  about  the  topic  and  file  the  notes  on  a floppy 
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disk.  Click  Audio  to  turn  the  audio  on  or  off.  Click  the  Calculator  to  access  a scientific  calculator  that  enables  the 
learner  to  compute  complex  formulas.  The  Tools  function  pad  is  designed  to  ensure  the  learner  has  access  to  the 
tools  needed  for  the  completion  of  the  lesson  at  all  times. 

At  the  bottom  of  the  screen  is  the  Navigation  Zone  that  contains  the  buttons  for  navigation  through  the 
module.  The  first  two  buttons  are  Quit  and  Main.  Click  on  Quit  to  return  to  the  Windows  desktop  and  exit  the 
module.  Click  on  Main  to  go  back  to  the  CM1  home  screen  to  select  other  activities  in  the  module  or  to  exit.  The 
module’s  on-line  activities  (topics,  self-checks,  and  assessment)  and  off-line  activities  (reading  assignments,  labs, 
case  studies,  etc.)  arc  located  at  the  Main  Menu  screen. 

Three  Application  Link  Navigation  buttons  are  provided  to  the  right  of  the  Main  button.  Link  buttons  will 
only  be  available  if  the  module  is  designed  to  have  the  student  access  an  external  program  (c.g.,  Excel)  during  the 
lesson.  A generic  button  space  is  reserved  for  special  buttons  (such  as  Example  to  bring  up  an  illustration  or 
example)  that  may  be  needed  by  different  lessons.  Interaction  navigation  is  provided  when  an  interaction  is 
displayed.  The  buttons  (Return,  Hint,  Answer,  and  Reset)  will  be  highlighted  for  use.  The  last  is  the  Check  button 
that  allows  the  user  to  access  a self-check  exercise.  Once  in  the  exercise,  the  button  changes  to  Score,  so  the  student 
can  click  on  it  to  access  his  or  her  numerical  score  for  the  Self-Check.  The  fourth  set  contains  Standard  Navigation 
buttons,  which  allow  the  user  to  screen  forward  and  back  one  screen  at  a time  and  to  go  all  the  way  to  the  beginning 
and  to  the  end  of  the  topic. 

The  Screen  Layout  has  four  basic  areas:  Screen  Title , Screen  Number , Medio  Area,  and  Text  Area.  Each 
screen  has  a unique  title  and  screen  number  that  is  designed  to  orient  the  learner  to  his  or  her  place  in  the  topic, 
which  is  an  important  instructional  aid.  This  screen  layout  also  is  designed  to  focus  the  learner’s  attention  and  to 
build  the  learner’s  confidence  because  it  increases  the  likelihood  of  the  learner  understanding  the  lesson  progression 
and  organization,  and  decreases  the  likelihood  of  the  learner  getting  lost  or  disoriented  (Park  & Hannifin,  1993). 

The  Media  Area  is  reserved  for  graphics,  animation  and  video,  and  is  generally  placed  at  the  left  of  the 
screen.  The  Text  Area  is  at  the  right  of  the  screen.  The  goal  is  to  keep  the  text  focused  on  one  main  point  with 
approximately  ten  lines.  Research  shows  that  consistent  screen  layout  with  understandable  organization  increases 
the  learner’s  learning  (Hannafin,  Hannafin,  Hooper  & Richer,  1996). 

Satisfaction 

Satisfaction  is  a category  that  concentrates  on  helping  learners  feel  positive  about  their  achievement.  This 
involves  combining  appropriate  external  rewards  with  the  challenge  of  providing  opportunities  to  achieve  internal 
rewards  in  the  areas  of  natural  consequences,  positive  consequences,  and  equity  (Keller  & Kopp,  1987). 

To  ensure  the  learner’s  satisfaction  with  the  interface  and  the  presentation  of  the  lesson,  the  CBI  team 
emphasized  user  control  with  navigation  options,  helpful  guidance  and  feedback,  as  well  as  equitable  assessment. 
The  learner  is  guided  through  practice  exercises  (interactions)  and  self-checks  with  access  to  more  information, 
hints,  and  solutions.  The  learner  may  go  through  a module  at  his  or  her  own  pace  and  may  take  the  self-check  as 
many  times  as  desired.  At  the  end  of  a module,  the  learner  takes  a module  assessment  that  is  matched  to  the 
objectives  and  the  self-checks.  Since  a Computer-Managed  Instruction  (CMI)  program  tracks  the  learner’s  activity, 
the  learner  can  access  his  or  her  scores.  Access  to  scores  is  designed  in  to  help  increase  the  learner’s  satisfaction 
with  the  lesson  because  it  provides  a sense  of  completion  and  an  intrinsic  reward. 

Collaborating  with  Faculty,  Using  the  ARCS  Model  to  Guide  Design  and  Development 

The  CBI  design  team  meets  with  the  faculty  developers  numerous  times  to  plan  the  modules.  The 
instructional  designer  is  the  primary  contact  with  the  faculty  developers  in  the  design  and  development  of  the 
courseware.  The  design  team  prepared  and  distributed  a process  handbook  and  a project  notebook,  containing 
electronic  and  paper  copies  of  planning  aids  and  storyboard  templates.  The  faculty  developers  arc  advised  to  use  the 
ARCS  Model  in  the  planning  of  instructional  strategies  and  content  presentation  of  the  screens.  Furthermore,  a 
checklist  is  provided  to  help  in  the  formative  evaluation  process. 

Implementing  the  Process 

Evaluation  of  the  instructional  product  while  it  is  in  development,  is  critical  to  its  successful  completion. 
Therefore,  the  design  team  incorporated  several  stages  and  types  of  formative  evaluation  to  ensure  that  the  prototype 
incorporated  the  desired  ARCS  elements,  followed  best  practices,  and  presented  a functional,  user  friendly  program 
interface.  Formative  evaluation  in  the  form  of  reviews  were  conducted  at  various  stages  with  five  users,  one  content 
expert,  one  multimedia  expert,  and  three  instructional  designers.  Evaluation  forms  based  on  the  ARCS  model  were 
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used  with  the  sample  learners  and  instructional  designers  who  reviewed  the  module  prototype.  Except  for  the  users, 
all  reviewers  were  experts  in  their  field  and  were  not  associated  with  the  design  team’s  efforts.  The  data  collected 
from  these  reviews  were  used  to  make  significant  changes  to  the  prototype  and  all  present  and  future  CBI 
courseware  designs  will  be  based  on  this  prototype. 

The  design  team  works  collaborativcly  with  the  faculty  developers  (approximately  60  from  five 
universities).  A primary  goal  is  to  keep  them  informed  of  the  basic  building  blocks  in  the  new  process.  First,  the 
design  team  conducted  two  workshops  for  this  purpose  in  the  past  year.  Second,  the  design  team  produced  and 
distributed  an  extensive  process  handbook  that  explains  the  coalition’s  mission  and  goals,  the  design  approach,  the 
collaboration  process,  meeting  agendas,  and  the  consistent  guidelines  for  the  design  and  development  of 
multimedia.  The  ARCS  model  is  a key  part  of  the  design  process  handbook.  Third,  to  ensure  that  the  faculty 
developers  have  ample  support  as  they  plan  and  design  their  course,  the  design  team  developed  over  thirty  planning 
aids  that  support  the  complex  process  of  multimedia  design. 

Central  to  these  planning  aids  arc  flow  charts  and  storyboard  templates.  These  serve  as  the  blueprint  for  the 
interactive  multimedia.  The  faculty  team’s  PI  (Principle  Investigator)  works  with  the  instructional  designer  to  lay 
out  each  screen  of  the  CBI.  The  instructional  designer,  and  other  members  of  the  team,  as  needed,  meet  with  the 
developers  as  often  as  twice  a month  to  confer  on  the  design  process,  which  requires  a lot  of  feedback  and  revision. 
When  final  versions  of  the  planning  aids  and  storyboards  are  complete,  the  developers  provide  paper  and  electronic 
copies  to  the  design  team  so  programming  can  start. 

In  1997,  the  CBI  design  team  accomplished  its  first  goal:  the  completion  of  a prototype  and  one  module  for 
one  course.  In  addition,  the  design  team  began  planning  modules  with  six  other  faculty  development  teams.  Two 
teams  have  completed  storyboarding  their  first  module.  In  October  1997,  the  prototype  was  presented  to  a NSF 
review  leant  and  given  high  ratings  with  a challenge  to  complete  25-30  modules  in  1998.  To  accomplish  this,  the 
design  team  will  double  its  staff,  adding  three  instructional  designers,  one  graphic  designer,  and  three  multimedia 
programmers.  It  will  be  a significant  challenge  to  achieve  this  rate  of  production.  Based  on  the  positive  reviews  by 
NSF,  the  users,  and  external  experts,  the  design  team  is  confident  that  the  established  prototype  and  planning 
process  that  incorporate  the  ARCS  Model  (Attention,  Relevance,  Confidence,  and  Satisfaction)  into  the  basic  design 
will  continue  to  provide  a solid  foundation  on  which  to  build. 

Conclusion 

The  design  and  development  of  engineering  courseware  is  a collaborative  process,  involving  the  Wayne 
State  University  Design  Team,  the  university  faculty  developers,  representatives  from  industry  and  Focus:HOPE. 
The  prototype  design,  which  is  based  on  the  ARCS  Model  Elements  of  Attention,  Relevance,  Confidence  and 
Satisfaction,  will  he  used  in  all  future  CBI  course  development.  While  still  under  development,  the  prototype  was 
reviewed  by  students  in  the  engineering  program  at  Focus:HOPE  as  well  as  by  experts  in  instructional  design  and 
multimedia.  The  data  collected  from  the  reviews  led  the  team  to  make  changes  in  the  screen  graphics,  colors,  button 
shapes  and  placement.  The  basic  screen  design  and  interface  were  given  favorable  reviews  for  their  ease  of 
navigation,  on-line  access  to  help,  and  high  level  of  user  control. 

The  challenges  that  face  the  design  team  and  the  faculty  developers  arc  many  as  we  work  together  to  use 
the  ARCS  Model  in  the  design  and  development  of  computer-based  college  engineering  courses.  Meeting  the 
production  schedule  is  one  of  the  most  pressing  challenges.  However,  the  following  challenges  are  also  important: 

• Designing  computer-based  lessons  that  are  of  sufficient  challenge  and  depth  for  a college  level  course. 

• Connecting  the  subject’s  conceptual  theory  to  relevant  applications. 

• Providing  sufficient  problem-based  learning  opportunities,  interaction,  feedback,  and  on-line  support  of  the 
lesson. 

• Maximizing  the  possibilities  of  multimedia  while  adhering  to  sound  learning  theory. 

• Staying  current  with  new  technology  and  translating  the  CBI  modules  to  other  instructional  delivery  media  such 
as  the  World  Wide  Web. 

• Determining  the  effectiveness  of  the  CBI  modules’  instruction  by  evaluating  the  students’  learning. 
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Abstract 

The  Internet  contains  a vast  array  of  educational  resources,  ranging  from  individual  activities  io  whole 
curricula.  This  research  study  used  content  analyses  and  an  electronic  questionnaire  to  investigate  how  and  why 
PreK-12  educators  use  information  from  the  Internet  for  instructional,  design  purposes  in  order  to  inform  the  design 
of  an  Internet-based  system  that  provides  one-stop,  any-stop  access  to  Internet-based  educational  materials.  Results 
indicate  that  lesson  plans  are  the  most  sought-after  instructional  resource  on  the  Internet.  The  data  also  revealed 
that  although  most  respondents  use  the  Internet  for  instructional  planning,  they  often  consult  several  resources 
(print,  electronic,  and/or  human)  and  use  or  adapt  the  information  to  meet  their  specific  instructional  needs. 

Introduction 

The  number  of  educational  resources  available  via  the  Internet-World  Wide  Wcb~-valuable  collections 
containing  text,  images  and  sounds,  existing  solely  in  electronic  form— is  increasing  exponentially  every  day 
(Wcibcl,  1995).  The  organization  of  these  materials  is  defined  by  the  indexes  provided  by  efj^^if  the  various 
electronic  locator  services  (c.g.  Yahoo,  Alta  Vista)  and  independent  online  information  providers  [c.g.  Educational 
Resource  Information  Clearinghouse  (ERIC),  Eisenhower  National  Clearinghouse,  NASA],  But  when  the  scope  of 
resources  becomes  extensive,  requiring  richer  records  for  each  resource  in  order  to  improve  search  and  retrieval,  the 
usefulness  of  these  Internet-based  indexes  becomes  limited  (Wcibcl,  1995). 

If  wc  think  of  each  of  these  electronic  collections  as  “virtual  libraries,1’  a teacher  could  currently  visit  any 
one  library  and  retrieve  some  materials  but  then  would  have  to  go  to  another  library  and  use  different  search 
strategies  to  find  other  materials  and  yet  another  for  more  or  different  materials.  However,  unlike  today’s  physical 
library  systems,  these  Internet-based  virtual  libraries  have  no  overall  organizing  structure  for  information  location 
and  access.  Extending  our  library  metaphor,  there  is  no  “union  catalog”  (i.c.  catalog  of  catalogs)  on  the  Internet  that 
allows  users  to  go  to  one  library  and  automatically  access  all  related  educational  materials  housed  in  other  library 
collections.  On  the  contrary,  the  Internet  user  must  enter  each  virtual  library  (assuming  an  awareness  of  ail  relevant 
electronic  collections  and  their  locations),  search  for  desired  information,  sift  through  the  located  resources  to 
determine  which  are  most  relevant  and  useful,  and  then  enter  another  virtual  library,  search  and  gather  information 
again,  enter  am,4Ucr  and  search  again,  and  so  on.  One  potential  result  is  information  overload;  for  example,  one 
participant  in  this  ‘r*v  reported  finding  more  than  30,000  lesson  plans  through  Tahoo  alone.  This  process  is  not 
only  cumbersome  and  linie-consuming,  but  also  discouraging  to  the  user,  frequently  causing  him  or  her  to 
prematurely  end  the  search  process  without  knowing  if  the  most  relevant  and  useful  information  has  been  located. 

The  Gateway  to  Educational  Materials  (GEM)  Project,  which  teams  a range  of  public  and  private 
information  providers,  national  educational  organizations,  regional  education  laboratories,  and  state  education 
departments,  was  funded  through  the  ERIC  Clearinghouse  On  Information  and  Technology  by  the  U.S.  Department 
of  Education  to  develop  a system  for  educators  that  provides  “one-stop,  any  stop  access”  to  the  broad  range  of 
educational  materials  (c.g.  teacher  guides,  lesson  plans,  curricula,  primary  documents)  on  the  World  Wide  Web. 
GEM  users  can  enter  any  of  the  educational  databases  within  the  GEM  system  (e.g.  the  AskERIC  Virtual  Library, 
Eisenhower  National  Clearinghouse,  NSF’s  Math  Forum),  specify  the  parameters  of  their  information  need  (c.g.  “a 
reading  curriculum  for  upper  elementary  gifted  students.”  “tenth  grade  history  lesson  plans  on  the  American  colonial 
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period0),  receive  a rich  set  of  records  describing  all  and  only  those  resources  that  meet  the  specified  criteria,  and 
obtain  quick  and  direct  access  to  those  resources. 

Each  resource  in  the  GEM  system  is  described  by  a simple,  standardized  metadata  record,  a set  of 
descriptive  terms  that  are  “more  informative  than  an  index  entry  but  less  complete  than  a formal  cataloging  record” 
(Wcibel,  1995,  p.l),  which  automated  tools  are  able  to  recognize  and  collect  (Weibel,  1995).  In  order  to  define  a 
metadata  profile  of  search  terms  that  simply  yet  sufficiently  represents  all  of  the  electronic  educational  resources 
contained  within  OEM’s  “union  catalog,”  research  was  conducted  to  explore  bow  and  what  instructional  resources 
arc  currently  represented  on  the  Internet,  what  types  of  information  educators  seek  when  designing  the  struction, 
and  how  educators  search  for  information  electronically. 

The  specific  research  questions  explored  were: 

(1 ) What  arc  the  most  common  types  of  instructional  resources  on  the  Internet  and  what  arc  their  essential 
elements? 

(2)  What  information  sources  (including  the  Internet)  do  educators  consult  for  instructional  planning? 

(3)  When  using  the  Internet  for  instructional  planning,  for  what  types  of  instructional  information  do  educators 
search? 

(4)  How  do  educators  rate  their  Internet  searches  in  terms  of  ease,  success,  and  satisfaction  with  the  search  process? 

Methods 

A variety  of  research  methods  was  utilized  to  answer  the  above  research  questions.  These  methods  were 
used  to  explore  teachers’  information-seeking  behaviors  when  performing  electronic  information  searches  on  the 
Internet  and  to  identify  the  essential  elements  of  instructional  resources  on  the  Internet.  The  methods  include: 

• • a content  analysis  of  Internet-based  instructional  resources  to  determine  their  most  common  elements.  As  part 
of  a course  assignment,  twenty-one  graduate  students  at  a northeastern  university  were  asked  to  locate  and 
retrieve  a minimum  of  five  instructional  resources  from  the  Internet-World  Wide  Web  for  analysis. 

• • a content  analysis  of  questions  submitted  to  AskERIC,  the  Internet-based  educational  question-and-answer 
service  of  the  ERIC  Clearinghouse  on  Information  & Technology,  to  explore  the  specific  instruction-related 
information-seeking  needs  of  educators  using  the  Internet  and  the  terms  they  use  to  describe  those  needs.  An 
analysis  of  1.995  questions  submitted  to  the  AskERIC  service  during  July  1996  was  performed  to  determine 
how  and  with  what  terms  educators  ask  for  information  for  instructional  planning.  The  advantages  of  this 
method  arc  that  it  ( 1)  is  an  unobtrusive  and  naturalistic  way  to  study  user  behavior  and  (2)  limits  bias  due 
to  reconstruction,  post-hoc  rationalization,  or  expectancy  effect. 

• • an  electronic  questionnaire,  based  on  the  data  gathered  from  the  content  analyses,  to  determine  how  educators 
search  for  instructional  information  on  the  Internet  and  administered  online  through  the  AskERIC  service. 
AskERIC  users  were  selected  because  they  were  more  likely  to  be  (1)  regular  users  of  technology  and  (2) 
members  of  the  targeted  preK-12  educator  audience.  Although  this  data  gathering  technique  was  easy  to 
administer  and  able  to  reach  large  numbers  of  potential  respondents,  there  were  disadvantages;  c.g.,  due  to  the 
length  of  the  questionnaire,  many  of  the  questions  were  necessarily  brief;  there  was  no  way  to  elicit  follow-up, 
clarifying  information. 

Results 

(1)  Whai  arc  the  most  common  types  of  instructional  resources  on  the  Internet  and  what  arc  their  essential 
elements? 

Educational  resources  range  in  scope  from  the  broadest  curricula  to  the  narrowest  activities.  Most  instructional 
resources  appear  to  fall  under  one  of  the  following  categories  (Eby  and  Kujawa,  1994): 

• curriculum  guides  or  courses  of  study  that  may  span  several  grade  levels  and  subject  areas  (c.g.  science,  U.S. 
History).  These  may  contain  a number  of: 

• units  , thematic  divisions  within  a semester  or  year-long  curriculum  (c.g.  constellations, 

American  Civil  War).  These  may  contain  two  or  more: 
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• lesson  plans,  one-  or  two-session,  organized  learning  experiences  (e.g.  the  Big  Dipper, 
the  Battle  of  Bull  Run).  The  narrowest  type  of  educational  resource  may  appear  as  part  of 
a lesson  plan  or  stand  alone  as  an  autonomous  entity.  These  include: 

• learning  resources  (e.g.  a rcfe.encc  book  on  constellations,  a multimedia  CD-ROM  on 
the  Civil  War) 

• learning  activities  (e.g.  making  a mobile  of  a constellation,  a field  trip  to  a Civil  War 
battleground) 

From  the  original  sample  of  1 1 1 Web-based  Internet  resources,  13  were  found  to  be  duplicates  and  three 
were  not  considered  instructional,  leaving  95  resources  for  analysis.  Of  the  95  resources,  there  were  72  lesson  plans 
(76%),  22  unit  plans  (23%),  and  one  activity  (1%).  Although  this  sample  was  relatively  small  in  comparison  to  the 
thousands  of  resources  available  on  the  Internet,  it  appears  to  be  representative,  indicating  that  lesson  pians  are 
among  the  most  common  type  of  instructional  resource  available  on  the  Internet. 

A content  analysis  of  the  95  Web-based  resources  revealed  32  categories  of  common  elements.  The 
elements  appeared  either  as  separate  and  distinct  elements  or  as  elements  embedded  within  other  elements.  Separate 
elements  appearing  in  90%  or  more  of  the  resources  were  activities  (95;  100%),  materials  (e.g.  handout,  textbook, 
video)  (91;  96%),  title  (88;  93%),  and  purpose  (e.g.  instructional  goals,  learning  objectives)  (87;  92%).  Elements 
embedded  within  other  elements  90%  or  more  of  the  time  were  instructional  strategies  (e.g.  examples)  (95;  100%), 
and  topic  (85;  90%)  (see  Table  1).  Elements  appearing  in  at  least  one-half  of  the  resources  were  grade,  subject  area, 
and  author  (separate)  and  grouping  (embedded).  Most  embedded  elements  were  found  within  either  title  or 
activities. 

In  order  to  further  explore  tbe  types  of  information  important  to  educators,  1 ,995  questions  submitted  to 
AskERIC  during  July  1996  were  retrieved  and  analyzed  for  relevance,  resulting  in  a total  of  161  relevant  [related  to 
prc-Kindcrgarten  (preK)  through  Grade  12  instructional  design]  questions  for  content  analysis..  PreK  was  included 
because  of  the  common  integration  of  preK  classes  into  elementary  schools  throughout  the  U.S.  (Riede,  1997).  Data 
were  analyzed  by  two  independent  coders  with  a .93  coefficient  of  reliability,  deemed  acceptable. 

Each  question  was  analyzed  to  determine  what  information  was  requested  and  what  information  was 
specified  within  it.  Data  analysis  determined  that,  by  far,  the  most  frequently  requested  type  of  instructional 
information  was  the  lesson  (54;  33%)  (sec  Figure  1).  All  other  requested  information  (e.g.  materials,  activities,  units) 
was  included  in  less  than  \0%  of  all  questions. 


403 


397 


Element 

Occurrence  N(%) 

Separate  n (%)  Embedded  n (%)  (Most  Common  Location) 

activities 

95  (100%) 

95  (100%) 

0 (0%) 

instate,  strategies  95  (100%) 

0 (0%) 

95  (100%) 

activities 

materials 

91  (96%) 

70  (77%) 

21  (23%) 

activities 

title 

88  (93%) 

88  (100%) 

0 (0%) 

purpose 

88  (93%) 

85  (98%) 

2 (2%) 

title/summary 

topic 

85  (90%) 

16(19%) 

69  (81%) 

title 

grade 

75  (79%) 

61  (81%) 

14(19%) 

title 

subject  area 

64  (67%) 

47  (74%) 

17(26%) 

title 

grouping 

56  (59%) 

8(14%) 

48  (86%) 

activities 

author 

56  (59%) 

56  (100%) 

0 (0%) 

date 

43  (45%) 

26  (61%<) 

17(39%) 

copyright  statement 

content  descrip. 

42  (44%) 

19(45%) 

23  (55%) 

activities 

author  address 

38  (40%) 

38  (100%) 

0 (0%) 

assessment 

37  (39%) 

28  (76%) 

9 (24%) 

forms 

time 

32  (34%) 

20  (63%) 

12(37%) 

activities 

enrichment 

32  (34%) 

30  (94%) 

2 (6%) 

activities 

publisher 

30  (32%) 

15  (50%) 

15(50%) 

copyright 

add.  resources 

27  (28%) 

25  (93%) 

2 (6%) 

activities 

materials  proced. 

19(20%) 

6(32%) 

13(68%) 

activities/materials 

copyright 

15(16%) 

15(100%) 

0 (0%) 

source 

13(14%) 

13(100%) 

0 (0%) 

forms 

19(10%) 

19(100%) 

0 (0%) 

prerequisites 

9(10%) 

9(100%) 

0 (0%) 

summary 

8 (8%) 

8(100%) 

0 (0%) 

other  audiences 

7(7%) 

7(100%) 

0 (0%) 

outline 

7 (7%) 

7(100%) 

0 (0%) 

location 

7 (7%) 

1 (14%) 

6 (86%) 

activities 

instruc.  style 

5 (5%) 

4 (80%) 

1 (20%) 

outline 

teacher  role 

4 (4%) 

4(100%) 

0 (0%) 

comments 

3 (3%) 

3(100%) 

0 (0%) 

season/wcather 

2 (2%) 

0 (0%) 

0(100) 

activities 

author  descrip. 

2 (2%) 

2(100%) 

0 (0%) 

Table  I.  Total  Occurrence,  Separate  and  Embedded  (with  Common  location)  Elements  Found  in  Instructional 
Resources  ( N~95 ). 
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Figure  1.  Requested  Information  in  AskERIC  Questions  (N=  161). 

Data  revealed  that  educators  typically  frame  their  queries  in  the  context  of  a subject  area,  i.c.  either  a broad 
curriculum  domain  such  as  mathematics  or  science  or  a more  specific  subject  such  as  algebra  or  ecology  (74;  46%). 
grade  range  (e.g.  middle  school,  primary)  (69;  43%),  and/or  topic,  i.c.  more  specific  subject  such  as  primary 
numbers  or  air  pollution  (57;  35%)  (sec  Figure  2).  A specified  grade  level  was  included  within  39  (24%)  of  the 
AskERIC  questions.  All  other  specified  elements  were  present  in  less  than  20%  of  all  queries. 
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Figure  2. 

(2)  What  information  sources  (including  the  Internet)  do  educators  consult  for  instructional  planning? 

A 32-item  questionnaire  consisting  of  21  multiple-choice  and  Likert-type  items  and  1 1 open-ended  questions  was 
emailed  to  2,  135  questioners  who  had  used  the  AskERIC  service  during  July  and  August  1996.  A total  of  283 


Known"  Information 


Specified  Information  in  AskERIC  Questions  ( N-I6! ). 


B 


{■»*  ^ J Hr#  L ’ 2 


y 

V 

9 


405  339 


responses  (13%)  were  returned;  23  were  classified  as  invalid  because  the  respondent  was  not  directly  involved  with 
preK-1  2 instruction,  leaving  260  questionnaires  for  analysis  The  low  response  rate  was  attributed  to  three  potential 
causes:  (1)  the  high  number  of  “other"  users  of  AskERIC  (e.g.  adult  educators,  school  administrators,  students)  who 
were  not  the  targeted  respondents  for  this  study,  (2)  some  AskERIC  users  may  have  submitted  more  than  one 
question  to  AskERIC  during  that  time  period  and,  therefore,  would  have  received  multiple  questionnaires  but  only 
returned  one,  and  (3)  the  difficulty  for  some  participants  to  respond  electronically  due  to  inadequate  hardware  or 
limited  access.  Results  where  N<260  indicate  missing  responses. 

Demographically,  respondents  (N=257)  repicr~nt  ?.!I  Lypcs  of  community  school  districts:  urban  (72;  28%), 
suburban  (92;  36%),  and  rural  (81;31%),  with  12  (5%)  indicating  they  came  from  some  combination  (e.g.  rural- 
suburban).  In  terms  of  years  of  instructional  experience,  more  than  one-half  (145;  57%)  of  respondents  (N=256)  had 
more  than  10  years  of  teaching  experience,  56  (22%)  had  6-10  years,  39  (15%)  had  been  teaching  from  one  and  five 
years,  and  16  (6%)  had  less  than  one  year  of  teaching  experience.  The  majority  (N=244)  were  female  (189;  77%), 
with  55  (24%)  male.  These  statistics  roughly  parallel  national  demographic  patterns  of  U.S.  educators  (NCES, 
1996;  1997). 

Respondents  also  represent  the  range  of  content  areas  of  preK-1 2 instruction.  Of  the  249  respondents,  22 
(9%)  were  from  the  math/scicncc  subjects;  31  (12%)  were  from  the  humanities  (language  arts,  English,  foreign 
languages,  religion  and  philosophy,  art);  16  (6%)  represented  the  social  sciences  (economics,  history,  geography, 
civics,  social  studies);  72  (29%)  taught  all  subjects  (e.g.  elementary,  special  education);  34  (14%)  taught  multiple 
subjects  (i.e.  two  or  more  subjects,  but  not  all  subjects);  and  74  (30%)  were  information  technology  educators 
(library  media  specialists,  computer  teachers)  (see  Figure  3).  The  over-representation  of  the  latter  category  [e.g. 
library  media  specialists  represent  2%  of  U.S.  educators  (NCES,  1996)]  is  most  likely  due  to  the  bias  of  drawing  the 
sample  from  the  AskERIC  user  population  which  typically  includes  a large  proportion  of  library  media  specialists 
and  technology  teachers. 
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Figure  3.  Content  Areas  Represented  by  Questionnaire  Respondents  (N=249). 

Although  an  attempt  was  made  to  classify  respondents  by  grade  level  categories  (e.g.  K-5,  9-12),  it  soon  became 
apparent  that  it  was  not  possible  to  create  exclusive  grade  level  clusters  because  of  the  wide  variety  of  responses  to 
this  question.  Some  responses  that  illustrate  this  variety  are:  “K-12,”  “grades  5-8,”  “10-14  year  olds,”  “grades  3,  5 
and  8,”  “preK-K,”  “grades  6- 1 2,”  and  “grade  1 .” 

Respondents  were  asked  to  rate  thirteen  information  resources  that  typically  contain  information  used  for 
instructional  design  on  a scale  of  1 (never  used)  to  5 (used  very  often).  In  general,  the  most  commonly  used 
resources  appear  to  be  print-based  (journals  & newsletters,  K-12  textbooks,  books/magazincs),  followed  by 
workshops  and  electronic  resources  (Internet  sites,  databases),  (see  Tabic  2).  When  it  comes  to  human  sources, 
colleagues  is  the  only  resource  ranked  above  3.0. 
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Source 

Ranking 

journals/newslctters 

3.81 

K-12  textbooks 

3.76 

books/magazincs 

3.74 

workshops 

3.64 

Internet  sites 

3.51 

electronic  databases 

3.46 

colleagues 

3.03 

librarians 

2.97 

experts 

2.91 

videos 

2.70 

bulletin  boards/iislservs 

2.70 

college  textbooks 

2.68 

radio/tv 

2.64 

Table  2.  Sources  of  Information  Used  for  Instructional  Planning  (] -never  used;  5= used  very  often)  (N-260). 

Respondents  also  indicated  that  they  use  a variety  of  sources  when  looking  for  information  to  design  their  lessons, 
preferring  the  “berrypicking  method”  as  described  by  Bates  (1989);  i.e.  consulting  several  sources,  vsclcciing 
“nuggets”  of  information  from  each  source  as  their  search  strategies,  questions,  goals,  and  relevance  criteria  evolve. 
Furthermore,  although  educators  describe  lesson  plans  and  other  instructional  resources  they  find  as  valuable,  they 
rarely  use  them  “as  is,”  but  rather  cither  adapt  them  to  their  specific  situation,  use  them  as  idea  generators  for  their 
own  plans,  or  use  part(s)  of  them. 

There  are  some  interesting  differences  among  groups  of  subject  area  educators  in  their  preferred 
information  sources.  Looking  at  the  top  five  ranked  sources  of  information  hy  subject  area,  joumals/ncwslctters  and 
K-12  textbooks  are  consistently  rated  first  and/or  second  most  important  across  subject  areas  (see  Table  3). 
However,  all  subjects  educators  ranked  colleagues  highest,  indicating  a higher  degree  of  dependence  on  other 
educators  for  instructional  planning  information. 


Math/Scicnce  (n=22) 

K-12  textbooks  (4.14) 
Workshops  (3.64) 
Colleagues  (3.55) 
Journals/magazincs  (3.45) 
Books/magazines  (3.4 1) 


Language/Arts  (n=31) 
Journals/newslctters  (3.67) 
K- 1 2 textbooks  (3.67) 
Books/magazincs  (3.67) 
Colleagues  (3.47) 

Internet  sites  (3.43) 


Social  Studies  (n=  1 6 ) 
Journals/Newsleltcrs  (4.29) 
K-12  textbooks  (4.21 ) 
Books/magazines  (4. 1 4) 
Internet  sites  (3.77) 
Colleagues  (3.50) 


Information/Tech.  (n=74)  All  Subjects  (n=72) 
Journals/magazincs  (3.93)  Colleagues  (3.93) 

Electronic  databases  (3.92)  K- 1 2 textbooks  (3 .79) 

Internet  sites  (3.91 ) Journals/magazines  (3.73) 

Books/magazines  (3.91)  Workshops  (3.65) 

Workshops  (3.85)  Books/magazines  (3.62) 


Multiple  Subjects  (n=34) 

K-12  textbooks  (3.97) 

Journals/magazincs  (3.77) 
Books/magazines  (3.69) 

Colleagues  (3.63) 

Internet  sites  (3.30) 


Table  3.  Top  Five  Rated  Preferred  Information  Sources  for  Instructional  Planning  by  Subject  Area  (N=249). 


Information/technology  educators  ranked  electronic  resources  (electronic  databases,  Internet  sites)  higher 
than  all  other  groups,  probably  because  this  group  is  more  likely  to  use  electronic  resources  on  a daily  basis.  Social 
science  educators  also  ranked  print  resources  (K-12  textbooks,  journals  & newsletters,  other  books)  higher  than  all 
other  groups  likely  due,  in  part,  to  the  importance  of  original,  print-based  historical  documents.  Internet  sites  were 
ranked  highly  by  humanities,  social  science,  informalion/tcchnology,  and  multiple  subjects  educators  but  not  by 
math/sciencc  or  all  subjects  educators.  Although  librarians  as  a human  resource  were  ranked  below  3.0  overall,  they 
were  rated  above  3.0  by  all  subjects  educators  (3.02)  and,  as  would  be  expected,  by  information/technology 
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educators  (3.52).  Radio/tv  was  rated  lowest  overall  except  by  social  science  educators,  possibly  due  to  the  usefulness 
of  these  media  for  obtaining  current  news  and  historical  events. 

Two  hundred-ten  respondents  (81%)  indicated  they  have  Internet  access  at  home,  while  193  (74%)  have 
access  at  school,  surpassing  the  national  average  (65%)  for  having  access  at  home  or  at  school  (NCES,  1997).  One 
hundred-fourteen  respondents  (44%)  reported  having  access  both  at  home  and  at  school.  High  access  rates  for 
respondents  in  this  study  are,  again,  likely  attributable  to  the  biased  sample  (AskERIC  users)  and  the  Internet  access 
requirement  for  participation  in  this  study.  Interestingly,  43  (17%)  of  those  who  stated  they  have  access  the  Internet 
at  school  indicated  they  never  use  the  Internet  at  school. 

When  asked  specifically  about  their  use  of  the  Internet  as  a resource  for  instructional  planning,  most 
respondents  (221;  85%)  indicated  that  they  use  the  Internet  often  (98;  40%)  or  sometimes  (1 13;  45%)  with  only  39 
(15%)  saying  they  never  use  the  Internet  for  designing  instruction.  A closer  look  at  these  data  by  subject  area 
revealed  that  social  science  educators  arc  the  most  frequent  users  of  the  Internet,  followed  by 
information/technology  educators  and  math/science  educators  (see  Figure  4). 


Figure  4.  Amount  of  Internet  Use  for  Instructional  Planning  by  Subject  Area  (N-240). 

(3)  When  using  the  Internet  for  instructional  planning,  for  what  types  of  instructional  information  do  educators 
search? 


Respondents  were  asked  to  rate  28  information  elements  on  their  importance  for  lesson  planning.  These 
elements  included  the  eight  common  terms  from  both  content  analyses  (purpose,  grade,  subject,  topic,  materials, 
grouping,  location,  and  assessment),  as  well  as  terms  derived  from  merging  similar  terms  (e.g.  instructional  style  and 
instructional  strategics)  and  adding  one  new  tcrm(standards)  because  of  the  recent  development  of  and  emphasis  on 
state  and  national  curriculum  standards.  Participants  were  asked  to  rate  each  term  was  rated  on  a scale  of  l=not 
important  to  5=  very  important.  Results  show  that  ten  elements  were  rated  4.0  or  higher,  including  topic,  subject, 
content  description,  materials  (resources  necessary  for  instruction),  forms  (e.g.  handouts,  worksheets),  grade 
lcvel(s),  purpose  (rationale,  goals,  objectives),  outline  of  lesson,  summary  and  assessment  (evaluation  of  student 
learning)  (see  Table  4).  Some  elements  that  “add  value”  to  educators’  information  searches  were  also  rated  above 
3.0  (e.g.  links  to  state  and  national  standards,  comments  from  colleagues  who  have  used  the  resource).  Of  the  28 
terms,  only  author  description  and  publisher  were  rated  below  3.0. 
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1 

1 

Element  ; N 

;Mean 

; Std  Dev 

Topic 

201; 

4.391 

0.62 

Subject 

202; 

4.36! 

0.71 

Content  description 

198j 

4.25. 

0.73 

Materials 

200; 

4.24 

0.77 

Forms 

.202; 

4.19 

0.81 

Grade 

203: 

4,16; 

0.86 

Purpose 

202. 

4.14 

0.82 

Outline 

203; 

4.06’ 

0.83 

Summary 

20 1; 

. 4.03 

0.82 

Assessment 

202 

4.03. 

0.88 

Materials  procedure  , 

199 

3.94 

0.93 

Additional  resources 

202 

3.92. 

0.82 

Prerequisite 

19  9. 

3.85, 

0.56 

Audience 

198. 

3.78: 

0.96 

Enrichment 

201; 

3.74 

0,94 

Instructional  style 

20  0: 

3.72; 

0.9 

Time  frame 

198; 

3.71. 

0.93 

Scope 

197; 

3.62 

1 .05 

Title 

201; 

3.45; 

1 .07 

Standards 

201. 

3.43. 

1 .03 

Grouping 

1 98; 

3.42 

1 

Comments 

202. 

3.26. 

1 .04 

Location 

“197 

3.24. 

1 .07 

Date 

194. 

3.19. 

1 .23 

Author  affiliation 

199. 

3.14; 

1.12 

Copyright 

197 

3.03. 

1.19 

Author  description 

198 

2.84 

1 .07 

Publisher 

198 

2.8 

1 .04 

Table  4 . Critical  Elements  of  Internet-Based  Lesson  Plans . 

The  questionnaire  used  a critical  incident  technique,  asking  respondents  to  recall  a recent  experience  in 
which  they  used  the  Intcrnet/World  Wide  Web  to  find  information  to  use  for  instructional  design  purposes.  Some 
examples  of  the  types  of  experiences  described  arc: 

• a middle  school  teacher  wanting  to  know  how  to  integrate  technology  into  an  interdisciplinary  language  arts/social 
studies  unit  on  slavery/emancipation  for  grades  6-8. 

• a high  school  French  teacher  looking  for  information  on  how  to  create  a kcypal  project. 

• a teacher  of  cducable  mentally  handicapped  high  school  teacher  seeking  a picture  of  the  human  heart  “to  use  with 
the  class  as  a visual  and  to  assist  in  the  discussion  of  the  human  heart.” 

• a computer  teacher  looking  for  computer-based  lessons  for  grades  prcK-6  on  a variety  of  topics  in  order  to 
develop  an  elementary  technology  curriculum. 

• a grade  1-2  teacher  looking  for  “sensory,  hands-on  activities  that  included  measuring  and  information  on  whales” 
fora  science  lesson. 
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• a high  school  English  teacher  searching  for  instructional  methods  and  materials  for  teaching  Shakespeare’s 
Macbeth. 

• a science  teacher  needing  information  on  ways  to  teach  students  in  grades  3 through  8 how  to  make  holograms. 

• an  8th  grade  civics  teacher  seeking  information  about  “the  positions  of  candidates  on  the  issues  in  terms  8th 
graders  could  understand.” 

Participants  were  asked  to  specify  which  Internet  resources  they  found  most  helpful  in  their  search.  Again  due  to  the 
biased  sample,  38  (43%)  of  total  respondents  (N-99)  cited  ERIC  or  AskERIC.  Other  specific  sites  mentioned  by 
more  than  one  respondent  were  NASA  Spacelink;  An  Aeronautics  and  Space  Resource  for  Educators 
(http://spacclink.msfc.nasa.gov/)  (7),  Texas  Education  Network(TENET)  (http://www,tenet.cdu)(4),  Carol  Hurst's 
Children’s  Literature  Newsletters  (http://www.carolhurst.com/newsletters/newsletters.htiril)  (2),  Kathy  Schrock’s 
Guide  for  Educators  (http://www.capccod.net/schrockguidc/)  (2),  Children’s  Literature;  A Newsletter  for  Adults 
(http:www.parcntsplace.com/readroom/  childnew/index.html)  (2),  and  U.S.  Department  of  Agriculture  (USDA) 
(http://www.usda.gov)  (2). 

(4)  How  do  educators  rate  their  Internet  searches  in  terms  of  ease,  success,  and  satisfaction  with  the  search  process? 

A series  of  subsequent  questions  asked  respondents  to  rate  their  recent  search  in  terms  of  the  case  of  the 
search  process,  the  amount  of  relevant  information  located,  and  their  satisfaction  with  the  information  found.  Of 
those  responding  (N=204),  slightly  more  participants  indicated  that  their  information  searches  were  somewhat  easy 
or  very  easy  (79;  39%)  than  those  who  rated  them  as  somewhat  difficult  or  very  difficult  (62;  30%),  while  63  (31  %) 
rated  their  searches  as  neither  easy  nor  difficult,  (see  Figure  5). 


Easy 

39% 


Neutral 

31% 


Figure  5.  Ease  of  Internet  Search  (N=204). 


A closer  look  by  subject  area  found  that  a majority  of  math/science  (59%)  and  social  science  (5S%) 
educators  described  their  searching  experiences  as  easy  (see  Figure  6).  Humanities  educators  were  the  only  group 
who  rated  their  searches  as  difficult  more  often  than  neutral  or  easy. 
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Figure  6.  Ease  of  Internet  Search  by  Subject  Area  (N~204). 


When  asked  how  much  relevant  information  was  found,  the  vast  majority  (178;  69%)  of  those  responding 
(N-260)  reported  finding  at  least  some  of  the  information  needed  [61(23%)  indicated  finding  most),  while  only  21 
(8%)  indicated  they  found  no  relevant  information  (sec  Figure  7). 


Figure  7,  Amount  of  Relevant  Information  Found  (N-260). 

Looking  at  amount  of  relevant  information  retrieved  by  subject  area,  social  science  educators  were  the  most 
successful  with  their  searches,  while  all  subject  educators  appear  to  be  least  successful  (sec  Figure  8). 


Figure  8.  Amount  of  Relevant  Information  Found  by  Subject  Area  (N-I99). 
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Finally,  most  respondents  (N=203)  rated  their  level  of  satisfaction  as  somewhat  satisfied  (82;  40%)  or  very  satisfied 
(72;  36%)  with  search  results,  while  18  (9%)  were  somewhat  dissatisfied  and  8 (4%)  were  very  dissatisfied.  Twenty- 
three  participants  (11%)  indicated  a neutral  response  (neither  satisfied  nor  dissatisfied)  (see  Figure  9). 


Figure  9.  Level  of  Satisfaction  with  Internet  Search  (N-203). 


Regardless  of  ease  or  search  or  amount  of  success,  most  educators  from  every  content  area  indicated  they  wv.rc 
satisfied  with  their  searches  (see  Figure  10). 
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Figure  10.  Level  of  Satisfaction  with  Internet  Search  by  Subject  Area  (N=202). 


Conclusions 

This  study  used  content  analyses  and  an  electronic  questionnaire  to  investigate  how  and  why  PreK-12 
educators  use  information  from  the  Internet  for  instructional  design  purposes.  Results  indicate  that  lesson  plans  arc 
the  most  sought  instructional  resource  on  the  Internet.  The  data  also  revealed  that  although  most  respondents  use  the 
Internet  for  instructional  planning,  they  often  consult  several  resources  (print,  electronic,  human)  and  use  or  adapt 
the  information  they  find  to  meet  their  specific  instructional  needs. 

Results  of  this  study  have  been  used  to  inform  the  design  of  the  GEM  system  by  identifying  the  most 
important  instructional  elements  in  educational  resources  (c.g.  topic,  grade,  forms,  assessment),  as  well  as  elements 
that  “add  value”  to  their  information  searches  (e.g.  standards,  comments).  These  elements  have  formed  the  metadata 
profile  that  describes  all  resources  included  in  the  GEM  system.  Future  research  is  planned,  including  wheth  r 
certain  metadata  elements  best  describe  the  resources  from  a particular  discipline  or  domain  and  whether  there  are 
changes  in  preferred  resources  (from  print  to  electronic)  as  GEM  provides  a more  responsive  Internet-based 
environment  for  meeting  the  instructional  planning  needs  of  educators. 
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To  be  in  the  world,  to  be  situated  at  all,  is  to  be  in  place. 
Place  is  the  phenomenal  particularization  of  ‘bcing-in-thc- world.’ 
Edward  Casey  (1993:  xv) 


The  Importance  of  Physical  Place  And  Lived  Topographies 

Michael  J.  Streibel 

The  University  of  Wisconsin  - Madison 


Abstract 

This  paper  is  motivated  by  a simple  question:  when  is  it  important  to  be  physically  present  in  teaching  and 
learning?  1 will  not  answer  this  question  directly  but  rather  will  articulate  a framework  for  thinking  about  the  role 
of  place  in  human  experience.  The  motivating  question  becomes  all  the  more  important  when  one  realizes,  as  does 
Joshua  Meyrowitz  in  No  Sense  of  Place , that  " electronic  media  destroy  the  specialness  of  place"  and  lead  to  a loss 
of  sense  of  place  ( Meyrowitz , 1985:  125).  and  when  one  realizes , as  does  Michael  Heim  in  Virtual  Realism,  that 
most  human  beings  "now  dwell  in  an  artificial  environment  ” which  is  on  the  verge  of  becoming  virtualized  by 
networked  electronic  media  (Heim,  1998:  153).  The  original  question  then  becomes:  why  is  physical  place  even 
important  in  our  experience?  I will  try  to  answer  this  latter  question  by  reviewing  recent  work  on  the  concept  of 
place  and  then  presenting  Keith  Basso’s  ethnographic  work  with  the  Western  Apache  on  their  belief  that  Wisdom 
Sits  in  Places  ( 1996).  1 will  end  by  sketching  out  further  research  questions  for  our  field  on  an  “ ethnography  of  lived 
topographies"  in  learning  places. 

Why  is  Physical  Place  Important  in  Our  Experience?  A Digression 

The  simplest  answer  can  be  stated  as  follows:  we  are  all  embedded  in  a physical  matrix  of  human  and 
“more-than-human”  beings  and  our  life  world  is  situated  in  the  here  and  now'.  Furthermore,  this  physical  matrix  plays 
a pervasive  role  in  the  construction  of  cultural  and  personal  identity,  so  that  the  more  we  become  distanced  from  this 
physical  matrix,  the  more  wc  become  distanced  from  the  grounds  of  our  lifeworld  and  ultimately  from  the  grounds 
of  our  being-in-thc-world.  Wc,  therefore,  have  to  understand  the  role  that  this  physical  matrix  plays  in  all  areas  of 
human  endeavor.  Since  physical  place  is  one  fundamental  aspect  of  our  immediate  physical  matrix  (our  bodies  arc 
another),  focusing  on  the  role  of  physical  place  in  our  experience  would  do  much  to  further  our  understanding.  To 
paraphrase  Wendell  Berry’s  famous  conclusion:  wc  don’t  know  who  we  arc  unless  we  know  where  wc  arc  (Stcgncr, 
1992:199). 

I was  led  to  this  conclusion  by  an  indirect  route.  1 have  critiqued  the  role  of  computers  in  education  for 
many  years  (Streibel,  1986)  and  have  attempted,  along  with  many  others  in  our  field,  to  reformulate  traditional 
theories  of  instructional  development  in  constructivist  directions  (Streibel,  1987,  1993,  1996).  This  led  me  to  look 
seriously  at  Lucy  Suchman’s  theory  of  situated  action  where  physical  place  played  a central  role  (Streibel,  1995). 
Contemporary  theories  of  instructional  development,  however,  had  very  little  to  say  about  the  physical  place  where 
learning  was  to  take  place  other  than  in  terms  of  ergonomic  factors  (Gustafson  & Branch,  1997).  This  current  state 
of  affairs  in  our  field  could  very  well  reflect  the  early  theoretical  influences  of  behaviorism,  communication  theory, 
and  systems  theory  as  Rita  Richey  has  shown  (Richey,  1986).  More  recent  developments  in  cognitive  theories  and 
information  processing  theories  refine  these  trends  by  focusing  on  what  happens  inside  the  “black  box”  of  the 
learner  but  little  attention  is  paid  to  the  physical  place  of  learning  as  a constitutive  component  of  learning  (Reigcluth, 
1983).  Constructivist  theories  of  instructional  development  moved  “outside  of  the  box”  by  focusing  on  the  social 
context  of  learning  and  the  social  construction  of  meaning  (Duffy  & Jonassen,  1992).  Recent  developments  in 
constructivist  theories  have  even  focused  on  the  design  of  whole  learning  environment  (Duffy  et.  al .,  1993)  and 
totally  virtual  classrooms  (Hiltz,  1994).  However,  the  physical  setting  of  learning  is  still  not  integrated  into  an 
ecology  of  learning  except  in  its  ergonomic  potential.  Speaking  of  education  in  general,  David  Orr,  in  Ecological 
Literacy,  says  that: 

other  than  as  a collection  of  buildings  where  learning  is  supposed  to  take  place,  place  has  no  particular 

standing  in  contemporary  education  (Orr,  1992:  126). 
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The  same  can  he  said  for  distance  education  where  “anytime,  anyplace"  in  “virtual  classrooms"  is  promised  (Evans 
& Nations  1989;  Keegan,  1996).  Perhaps  some  very  deep  theoretical  assumptions  are  still  very  much  in  force  even 
though  specific  theories  of  instructional  development  have  progressed  very  much.  Perhaps  the  time  has  come  to 
revisit  some  of  these  very  deep  assumptions.  I contend  that  the  concept  of  place  is  one  of  these  deep  theoretical 
assumptions  that  needs  to  be  revisited  in  order  to  further  our  understanding. 

Why  is  Physical  Place  Important?  Some  Background 

Let  me  situate  my  initial  answer  with  some  background.  A number  of  scholars  such  as  Edward  Casey  arc 
calling  for  a “renewed  understanding  of  the  place-world"  as  a matter  of  global  survival  (Casey,  1993).  David  Orr.  in 
Ecological  Literacy,  claims  that: 

education  relevant  to  the  transition  to  a sustainable  society,  demands  first,  an  uncompromising  commitment 
to  life  and  its  preseiyation  (Orr,  1992;  133). 

Furthermore,  real  learning  “foslcr[s]  a sense  of  connectedness,  implicatedness,  and  ecological  citizenship"  and 
entails  a “dialogue  with  a place"  (Orr,  1992:  1 03,  90).  Other  researchers,  such  as  C.  A.  Bowers,  on  the  other  hand, 
have  shown  how  our  use  of  technology  is  ecologically  unsustainable  because  it  embodies  cultural  values  that 
separate  us  for  the  world  and  east  us  in  the  role  of  controlling  and  dominating  the  materia'  world  (Bowers,  1988, 
1993,  1994).  These  contemporary  cultural  values  reflect  a kind  of  psychological  modernism  that  is  so  aptly 
described  by  Thomas  Moore  in  his  book  Care  of  the  Soul  (1992).  Psychological  modernism  is: 

an  uncritical  acceptance  of  the  values  of  the  modern  world.  It  includes  blind  faith  in  technology,  inordinate 
attachment  to  material  gadgets  and  conveniences,  uncritical  acceptance  of  the  march  of  scientific  progress, 
devotion  to  electronic  media,  and  a life-style  dictated  by  advertising.  This  orientation  toward  life  also  tends 
toward  a mechanistic  and  rationalistic  understanding  of  matters  of  the  heart  (Moore,  1992:  206). 

Moore  continues  by  saying  that  when  “technology  becomes  the  root  metaphor  ....  there  sometimes  seems  to 
be  an  inverse  relationship  between  information  and  wisdom  (Moore,  1992:  206-7).  Moore’s  response  is  to  propose  a 
reconnection  with  the  physical  particularities  of  person  and  place  as  a way  to  learn  about  “fate,  time,  nature, 
morality,  and  character ...  [things  that  arc]  important  in  life”  (Moore,  1992:  216). 

A simple  way  to  characterize  this  loss  of  connection  with  place  engendered  by  psychological  modernism  is 
to  compare  “digital  living,"  as  described  by  Nicholas  Negrponte  in  Being  Digital  (1995)  and  William  Mitchell  in 
City  of  Bits  (1995),  with  “embodied  living,"  as  described  by  Thomas  Moore  in  Care  of  the  Soul  (1992)  and  David 
Abram  in  The  Spell  of  the  Sensuous  (1996).  The  comparison  comes  down  to  a difference  in  privclcdging  the  abstract 
over  the  concrete.  For  example,  it  comes  down  to  a difference  between  (Strcibel,  1997): 

1.  abstract  representations  of  place  (or  explicit  knowledge  of  the  world  - e.g.,  virtual  places,  digital 
representations,  etc.)  vs.  embodied  engagement  with  specific  places  (i.c.,  tacit  knowledge  of  the  world). 

2.  abstract  representations  of  self  and  other  (e.g.,  data  structures,  avatars,  etc.)  vs.  embodied  lifcworlds  of  self  and 
other. 

3.  abstract  representations  of  interactions  (e.g.,  intelligent  tutoring  systems,  automated  workflow  systems,  etc.)  vs. 
physically-embodied  cultural  lifcworlds. 

1 have  cast  these  distinctions  in  either-or  terms  in  order  to  highlight  their  differences.  However,  if  we  arc  to 
frame  a concept  of  place  that  furthers  our  understanding  of  both  the  technological  and  the  natural  realms,  then  we 
have  to  come  up  with  a framework  that  encompasses  both  and  docs  justice  to  both.  We  have,  in  other  words,  to 
avoid  either-or  thinking  and  combine  the  abstract  with  the  concrete  in  our  thinking  and  actin''  This  means  both  an 
avoidance  on  over-reliance  on  abstractions  and  an  avoidance  of  reifying  place.  I believe  that  < .elational  concept  of 
place  will  help  us  in  this  effort.  There  are  many  proposals  about  how  to  proceed.  I will  only  sketch  out  a few  such 
proposals  and  show  what  they  add  to  a relational  concept  of  place  before  proceeding  to  Basso’s  work  which  I 
believe  holds  the  most  promising  ideas. 

Michael  Heim  in  both  his  earlier  work  on  The  Metaphysics  of  Virtual  Reality  (1993)  and  his  current  work 
on  Virtual  Realism  (1998)  argues  that  technologically-mediated  virtual  worlds  and  the  natural  world  (or  the  primary 
world,  as  he  calls  it)  make  different  demands  on  our  bodies  so  that  we  basically  have  to  become  competent  at 
operating  in  two  worlds.  Hence,  we  have  to  learn  how  to  become  totally  immersed  in  virtual  work  (and  learning) 
environments  and  then  develop  a whole  series  of  “decompression"  rituals  such  as  'iai  Chi  (Chuan)  and  Chi  Kung  in 
order  to  “reclaim  the  integrity  of  conscious  life  in  a biological  body"  (Heim,  1998:  172).  These  biologically- 
embodied  physical  rituals,  in  effect,  rcsituatc  our  consciousness  in  our  body’s  physical  place.  Heim’s  proposals  are  a 
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slep  in  the  right  direction  by  focusing  on  the  role  of  our  physical  body  in  achieving  a reconnection  with  the  physical 
world.  However,  Heim’s  proposals  still  set  up  physical  and  virtual  place  in  opposition. 

Other  authors  make  more  radical  proposals.  Let  me  give  you  one  example  from  Edith  Cobb  in  her  work  on 
The  Ecology  of  Imagination  in  Childhood  (Cobb,  1977).  Although  her  work  predates  the  widespread  use  of 
computers,  her  work  with  children  is  a prime  example  of  reconnecting  with  the  physical  world.  Cobb  recounts  the 
story  of  a young  girl  who  was  extremely  disassociated  from  her  body  and  her  world  except  when  she  went  on 
summer  vacations  in  Michigan.  The  young  girl  wrote  in  her  diary  as  she  was  recovering  by  reliving  the  memory  and 
imagery  of  her  summer  world: 

Big  trees,  pine  trees  swishing  in  the  wind.  The  Mountain,  blue  and  hazy  in  the  distance,  grass  blowing 
below  the  waves  on  die  ocean.  Sticky  bark,  blue  jeans,  that  was  Michigan  - and  tennis,  lots  of  it,  every  day. 
Then  a cool  swim,  dive  off  the  dock  and  the  water  rushing  by,  swimming,  lying  on  the  raft,  picnics  on  the 
beach,  the  Point,  singing,  and  marshmallows,  sweet  and  black  (Cobb,  1977:  76). 

What  you  sec  here,  as  Cobb  concludes,  is  a young  girl  whose  “body  image  and  world  image  had  conjoined, 
giving  a sense  of  true  identity”  (Cobb,  1977,  76).  Cobb  also  concludes  that: 

individual  health  is  a successful  adaptation  to  the  cosmos  as  well  as  to  culture  and  society  and  that  this 
adaptation  is  temporal  and  well  as  spatial  - [a  mutual  adaptation  that  can]  best  [be]  described  by  the  word 
‘ecological’  (Cobb,  1977:74-5). 

The  world  of  a specific  physical  place  and  the  imaginative  memor>  of  that  place  formed  a both-and  relation 
which  led  to  physical  and  mental  well-being.  Furthermore,  true  identity  entailed  a particularist  conjoining  of  the 
physical  and  social  worlds. 

The  idea  that  physical  place  has  an  impact  on  our  mental  well  being  is  becoming  a major  focus  in 
Architectural  design.  Since  the  1960’s,  architects  such  as  Christopher  Alexander  have  begun  to  show  that 
habitability  in  specific  physical  places  is  part  of  a deep  “ pattern  language ” of  all  living  beings,  and,  therefore,  calls 
for  a “timeless  way  of  building”  (Alexander,  1977,  1979).  A key  aspect  of  this  timeless  pattern  language  is  a 
“quality  [which]  is  objective  and  precise,  but  [which]  cannot  be  named.”  However, 

in  order  to  define  ,his  quality  ...  we  must  begin  by  understanding  that  every  place  is  given  its  character  by 
certain  patterns  of  events  which  keep  on  happening  there  (Alexander,  1 979:  ix-x). 

Patterns  of  living  in  a place,  rather  than  a reified  quality,  become  the  grounds  of  objectivity.  As  to  naming. 
Basso  will  show  how  the  Western  Apache  culture  succeeded  in  naming  such  a quality  with  place-names. 

Architects  also  began  to  make  the  case  that  architecture  is  a form  of  healing  because  of  the  way  that  people 
were  influenced  by  their  physical  places.  For  example,  Christopher  Day,  in  Places  of  the  Soul  (1990),  calls  for 
designing  physical  places  with  “health-giving  intent”  because  “our  environment  is  part  of  our  biography”  (Day, 
1990:  23).  Day  is  not  arguing  that  a physical  place  has  a reified  “spirit”  which  somehow  magically  fosters  healing. 
Rather: 

where  the  environment  can  offer  interest,  activity  and  intriguing  ambiguity,  timeless  durability  and  a sense 
of  roots  (in  place,  past  and  future)  in  the  wider  natural  world  with  its  renewing  rhythms  ....  it  can  provide 
support  as  the  first  step  to  recovery  (Day,  1 990:  26). 

How  can  we  approach  virtual  environments  so  that  they  become  part  of  our  biography  and  history  in  a 
health-giving  way  - especially  in  light  of  Bower’s  and  Heim’s  claim  that  such  environments  distance  us  from  our 
physical  bodies  and  the  physical  places  we  inhabit?  I am  only  asking  this  question  rhetorically  at  this  point  because  I 
want  to  develop  a framework  that  can  begin  to  address  this  question.  One  thing  is  clear,  however.  Any  new 
framework  has  to  begin  by  “embracing  the  real”  and  coming  to  terms  with  our  relationship  to  physical  places. 
Chcllis  Glcndinning  gives  a deep  reason  why  this  should  be  the  ease.  In  her  book.  My  Name  Is  Chellis  and  I'm  In 
Recovery  from  Western  Civilization , she  writes  that: 

our  primal  matrix  grew  from  the  Earth,  is  inherently  part  of  the  Earth,  and  is  built  to  thrive  in  intimacy  with 
the  Earth  (Glcndinning,  1994:  16). 
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Olhers  echo  her  concern.  Charlene  Spremak,  in  The  Resurgence  of  the  Real , argues  that: 

a truly  pas/modern  alternative  would  counter  the  modern  ideological  Right  from  body,  nature,  and  place 
(Sprctnak,  1997:  223). 

Hence,  a truly  postmodern  alternative  framework  would  be  an  “ecological  postmodernism ” that  focuses  on 
nature,  body,  and  place.  The  rest  of  this  paper  focuses  primarily  on  place  although  the  other  elements  arc  never  far 
away. 

The  idea  of  “embracing  the  real”  is  not  without  its  dangers  because  it  could  lead  us  back  down  a path  of 
opposing  the  abstract  and  the  concrete  in  an  either-or  fashion.  Hence,  wc  could  claim,  as  docs  Bill  McKibbcn  in  The 
Age  of  Missing  Information  (1992),  that  real  information  can  only  flow  into  us  through  unmediated  experience. 
(McKibt-cn  compared  watching  24-hours  worth  of  many  TV  channels  over  a several  month  period  with  a twenty- 
four  hour  period  out  in  the  Adirondacks).  However,  both  mediated  and  unmediated  (i.c.,  technologically-mediated 
and  embodied)  experiences  are  “real”  in  the  sense  of  becoming  part  of  our  personal  and  social  biographies.  Focusing 
on  how  wc  pcrccplually-engagc  other  beings  no  matter  what  the  manner  and  how  this  contributes  to  our  personal 
and  social  biographies,  therefore,  seems  a better  and  more  fruitful  path  to  follow.  This  docs  not  discount  the  real 
differences  between  mediated  and  embodied  experiences  but  it  helps  us  “move  out  of  the  map,  [and]  into  the 
territory,’'  to  use  David  Abram’s  felicitous  phrase,  without  jettisoning  the  map  (Abram,  1995:  97). 

As  to  arguing  for  the  primacy  of  engaging  the  physical  world,  Abram,  in  his  book  The  Spell  of  the 
Sensuous(  1996),  uses  Merlcau  Ponty’s  notion  of  “perceptual  reciprocity”  to  claim  that  both  our  physical  bodies  and 
the  physical  world  are  always  required  for  meaning. 

The  world  and  I reciprocate  each  other,  [he  claims,  such  that]  our  spontaneous  experience  of  the  world  (is] 
charged  with  subjective,  emotional,  and  intuitive  content  [that]  remains  the  vital  ground  ...  of  all  our 
objectivity  (Abram,  1996:  33-34). 

This  “vital  ground  ..  of  all  our  objectivity”  will  be  a key  component  of  a relational  concept  of  place. 
Arnold  Bcrlcant.  Living  in  the  Landscape , comes  a similar  conclusion.  “The  desire  for  a congenial  integration  of 
body  and  environment,”  he  writes,  “is  expressed  in  a yearning  for  a sense  of  place”  (Bcrlcant,  1997:  108).  (Bcrlcant 
also  says  that  “continuity  (of  body  and  environment]  is  not  always  positive”  and  cites  the  Pruilt-Igoc  Housing 
Project  in  St.  Louis.  1997:109). 

The  idea  of  a “sense  of  place”  in  terms  of  specific  local  particularities  is  linked  by  other  authors  to  ideas  of 
community  and  the  social  construction  of  meaning.  Tony  Hiss,  in  The  Experience  of  Place , has  shown  how  a sense 
of  place 

allows  any  of  us  a direct  sense  of  continuing  membership  in  our  communities,  and  our  regions,  and  the 
fellowship  of  all  living  creatures  (Hiss,  1990:  xiii). 

David  Abram  expands  the  notion  of  communities  to  encompass  thc”morc-lhan-human-world”  of  living  and 
non-living  beings  (such  as  whole  eco-systems).  In  each  of  these  cases,  personal  and  communal  identity  is  involved 
in  our  engagement  with  the  physical  world.  This  conclusion  is  made  explicit  by  Daniel  Kcmmis  in  Community  and 
the  Politics  of  Place  where  he  argues  that 

public  life  can  only  be  reclaimed  by  understanding,  and  then  practicing,  its  connection  to  real,  identifiable 
places  (Kcmmis,  1990:  6). 

Kcmmis’  ultimate  claim  is  that  “no  real  culture  can  exist  in  abstraction  from  place”  (Kcmmis,  1990:  7).  I 
would  only  add  that  no  real  culture  can  exist  in  abstraction  alone. 

Hence,  wc  come  full  circle  to  the  premise  of  this  paper,  namely,  that  a new  framework  for  how  wc 
construct  meaning  has  to  start  with  our  experience  of  physical  place.  1 would  now  like  to  turn  to  Keith  Basso’s  work 
with  one  native  American  nation  - the  Western  Apache.  My  reason  for  choosing  this  line  of  inquiry  is  because  the 
Western  Apache  embody  a relational  concept  of  place  in  their  daily  practice.  This  is  not  to  suggest  that  we  become 
like  the  Western  Apache  or  that  wc  appropriate  their  cultural  constructions  for  our  own  purposes  and  interests. 
Rather,  there  is  much  we  can  learn  from  them.  Although,  as  Bcrlcant  says  in  Living  in  the  Landscape, 

the  native  North  American  peoples  often  identified  their  bodies  in  a literal  sense  with  the  land  and  with 
other  things,  living  and  non-living,  whose  home  is  that  land, 
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they  also  hold  a view  that  secs 

humans  as  an  interrelated  part  of  the  natural  world  and  that  world  as  a congruent  whole,  all  parts  of  which 
possess  value  and  deserve  respect  (Berleant,  1997:  99). 

It  is  this  latter  commitment  which  I would  like  to  retain  as  I develop  a relational  concept  of  place  and 
eventually  an  “ethnography  of  lived  topographies”  for  our  field,  to  use  Basso’s  term  for  his  methodology  (Basso, 
1996:  1 10).  Berleant  is  even  more  explicit  about  this  commitment  and  its  relevance  to  a concept  of  place,  Our 
identity  as  beings  in  a world  of  living  and  non-living  beings , he  claims,  is  “ bound  up  with  its  physical  place  of 
inhabitation  ” (Berleant,  1997:  100). 

Wisdom  Sits  in  Places:  An  Example  of  a Relational  Concept  of  Place 

Keith  Basso  spent  fifteen  years  living  and  working  with  the  Western  Apache  in  Cibccue,  Arizona.  Then, 
after  being  asked  by  tribal  elders,  he  mapped  out  the  living  presence  of  physical  places  in  their  community.  These 
places  were 

places  not  far  away  ...  places  made  memorable,  and  infinitely  imaginable,  by  events  that  happened  long  ago 
when  the  people’s  distant  ancestors  were  settling  into  the  country  (Basso,  1996:  8). 

The  maps  of  these  places  were  not  drawings  with  Cartesian  coordinates  but  maps  of  the  eve  and  the  mind. 
The  maps  contained  place-names  that  were  “bold,  visual,  [and]  evocative”  and  stories  of  memorable  events  in  those 
places  that  contained  lessons  for  right  action.  Although  place-names  were  used  as  “situating  devices”  in  all  forms  of 
stories,  they  were  used  in  historical  tales  as  “instruments]  of  survival”  (Basso,  1996:  131 ).  An  example  of  a general 
situating  place-name  would  be  “Water  Flows  Inward  Under  a Cottonwood  Tree.”  An  example  of  a place-name  with 
an  historical  tale  is  “Men  Stand  Above  Here  and  There.”  Let  me  use  Basso’s  own  words  to  describe  how  the  latter 
place-name  was  used  in  a contemporary  situation: 

In  early  June  1977,  a seventccn-ycar-old  Apache  woman  attended  a girl’s  puberty  ceremonial  at 
Cibecue  with  her  hair  rolled  up  in  a set  of  pink  plastic  curlers.  She  had  returned  home  two  days  before  from 
a boarding  school  in  Utah  where  this  sort  of  ornamentation  was  considered  fashionable  by  her  peers. 
Something  so  mundane  would  have  gone  unnoticed  by  others  were  it  not  for  the  fact  that  Western  Apache 
women  of  all  ages  are  expected  to  appear  at  puberty  ceremonials  with  their  hair  worn  loose.  This  is  one  of 
several  ways  that  women  have  of  showing  respect  for  the  ceremonial  and  also,  by  implication,  for  the 
people  who  have  staged  it.  The  practice  of  presenting  oneself  with  free-flowing  hair  is  also  understood  to 
contribute  to  the  ceremonial’s  effectiveness,  for  Apaches  hold  that  the  ritual’s  most  basic  objectives,  which 
arc  to  invest  the  pubescent  girl  with  qualities  necessary  for  life  as  an  adult,  cannot  be  achieved  unless 
• standard  forms  of  respect  are  faithfully  observed.  On  this  occasion  at  Cibecue,  everyone  was  following 
custom  except  the  young  woman  who  arrived  wearing  curlers.  She  soon  became  an  object  of  attention  and 
quiet  expressions  of  disapproval,  but  no  one  spoke  to  her  about  the  cylindrical  objects  in  her  hair. 

Two  weeks  later,  the  same  young  woman  made  a large  stack  of  tortillas  and  brought  them  to  the  camp 
of  her  maternal  grandmother,  a widow  in  her  mid-sixties  who  had  organized  a small  party  to  celebrate  the 
birthday  of  her  eldest  grandson.  Eighteen  people  were  on  hand,  myself  included,  and  all  of  us  were  treated 
to  hot  coffee  and  a dinner  of  boiled  beef  and  potatoes.  When  the  meal  was  over,  casual  conversation  began 
to  flow,  and  the  young  woman  seated  herself  on  the  ground  next  to  her  younger  sister.  And  then  - quietly, 
deftly,  and  quite  without  warning  - her  grandmother  narrated  a version  of  the  historical  talc  about  the 
forgetful  Apache  policeman  who  behaved  too  much  like  a whiteman.  Shortly  after  the  story  was  finished, 
the  young  woman  stood  up,  turned  away  wordlessly,  and  walked  off  in  the  direction  of  her  home.  Uncertain 
of  what  had  happened,  I asked  her  grandmother  why  she  had  departed.  Has  the  young  woman  suddenly 
become  ill?  “No,”  her  grandmother  replied.  “I  shot  her  with  an  arrow.” 

Approximately  two  years  after  this  incident  occurred,  I found  myself  in  the  company  of  the  young 
woman  with  the  taste  for  distinctive  hairstyles.  She  had  purchased  a large  carton  of  groceries  at  the  trading 
post  at  Cibecue,  and  when  I offered  to  drive  her  home  with  them  she  accepted.  I inquired  on  the  way  if  she 
remembered  the  time  that  her  grandmother  had  told  us  the  story  about  the  forgetful  policeman.  She  said  she 
did  and  then  went  on,  speaking  in  English,  to  describe  her  reactions  to  it.  “I  think  maybe  my  grandmother 
was  getting  after  me,  but  then  I think  maybe  not,  maybe  she’s  working  on  somebody  else.  Then  I think 
back  on  that  dance  and  I know  it’s  me  for  sure.  I sure  don’t  like  how  she’s  talking  about  me,  so  I quit 
looking  like  that.  I threw  those  curlers  away.”  In  order  to  reach  the  young  woman’s  camp,  we  had  to  pass 
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within  a few  hundred  yards  of  Men  Stand  Above  Here  and  There,  the  place  where  the  man  had  lived  who 
was  arrested  for  rustling  in  the  story.  I pointed  it  out  to  my  companion.  She  said  nothing  for  several 
moments,  Then  she  smiled  and  spoke  softly  in  her  own  language:  “I  know  that  place.  It  stalks  me  every  day 
(Basso,  1990:56-57). 

What  we  have  here  arc  all  the  elements  of  place  in  Western  Apache  culture: 

• place-names  (i.c.,  Men  Stand  Above  Here  and  There)  that  tic  events  in  a narrative  to  physical  places. 

• stories  that  are  “concerned  with  disruptive  social  acts  ...  [and]  a stark  reminder  that  trouble  would  not  have 
occurred  if  people  had  behaved  in  ways  they  knew  they  should”  (Basso,  1996:  J8). 

• place-names  that  give  the  listener  “pictures  to  work  with”  rather  than  moral  injunctions. 

• physical  places  that  “stalk”  the  listener  and  “proclaim  by  their  presence  and  their  names  both  the  immanence  of 
chaos  and  the  preventive  wisdom  of  moral  norms”  (Basso,  1996:  28). 

All  of  the  elements  of  place,  person,  situation,  and  communal  historical  story  arc  fused  into  a semiotic 
relationship.  Furthermore,  each  element  changes  as  a result  of  this  fusion: 

• the  specific  physical  place  becomes  an  individual’s  pcrsonally-storied  place.  It  is  henceforth  permanently 
experienced  in  a different  way. 

• the  specific  person  becomes  more  linked  with  the  historical  community  whether  he  or  she  chooses  to  heed  the 
“wisdom”  of  the  historical  story. 

• the  specific  situation  becomes  a unique  instance  of  an  historical  story. 

• the  historical  story  is  both  reinforced  and  enriched  for  future  use  by  how  the  specific  individual  makes  it  part  of 
their  personal  biography. 

In  an  ironic  (to  us)  way,  this  is  “history  without  authorities”  that  simultaneously  honors  the  “wisdom”  of 
the  elders.  Physical  place  plays  a crucial  role  here  because  it  is  a relatively  stable  witness  by  it  presence.  However, 
physical  place  can  only  play  this  role  in  communal  sense-making  (and  survival)  when  individuals  form  an  “enduring 
bond  with  [these]  physical  places.”  Place  is  not  an  absolute,  external  reality  but  a very  real  and  objective  part  of 
everyday  experience. 

The  reason  that  physical  place  can  play  such  a central  role  in  everyday  experience  as  well  as  in  personal 
biography  and  communal  history  is  because 

the  constructions  Apaches  impose  upon  their  landscape  have  been  fashioned  from  the  same  cultural 
materials  as  constructions  they  impose  upon  themselves  as  members  of  society  (Basso,  1996:  102). 

These  “cultural  materials”  arc  only  dependent  on  an  identification  of  person  and  place  in  a 
phenomenological  sense.  Physical  places,  in  effect,  are  not  “brought  fully  into  being”  until  a biography  and  history 
of  lived  relationships  has  accrued  in  that  place.  In  the  historical  place-name  example  above,  a hundred-year-old 
story  of  cultural  resistance  in  a specific  place  was  used  in  a way  that  the  intended  hearer  of  the  story  would 
henceforth  take  very  personally.  Only  then  could  the  physical  place  “speak”  to  the  hearer.  Only  then  could  the 
physical  place  “stalk”  the  hearer.  The  emotional  and  meaningful  context  of  such  “speaking”  and  “stalking”  would 
come  from  the  fusion  of  hearer,  community,  and  place.  As  Basso  concludes,  “selfhood  and  placehood  arc 
completely  intertwined”  (Basso,  1996:  146). 

The  use  of  place-names  in  Western  Apache  historical  stories  entails  a “running  exchange  of  depicted 
pictures.”  At  first  sight,  this  might  seem  like  an  act  of  visual  memory  - much  like  Yates  described  in  The  Art  of 
Memory  (1966).  However,  the  Western  Apache  use  of  place-names  goes  much  further.  Each  person  in  the 
“exchange  of  depicted  pictures”  helps  construct  a unique  interactive  communication  for  the  hearer.  The  hearer,  in 
turn,  has  to  “add  on"  by  first  drawing  on  his  or  her  own  memory  of  specific  places  and  stories  and,  then,  by 
reflecting  on  how  these  places  and  their  stories  apply  to  their  own  unique  situation.  The  cultural  construction  of 
meaning,  therefore,  plays  out  in  both  the  community  and  in  each  individual.  Physical  place  plays  a key  role  here 
because  it  is  an  enduring  placc/cvcnt  which  shapes  the  life  of  the  community  as  well  as  the  life  of  the  individual. 
Basso  writes  that 

if  the  message  is  taken  to  heart  by  the  person  at  whom  the  tale  is  aimed  ...  a lasting  bond  will  have  been 
created  between  the  individual  and  the  site  ....  [once  this  has  happened!  the  features  of  the  physical 
landscape  take  over  and  perpetuate  [the  bond]  (Basso.  1996:  55,  60). 
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The  uniqueness  and  abidingness  of  place,  in  effect,  contributes  to  the  uniqueness  and  abidingness  of  personal  and 
communal  meaning. 

There  is  much  we  can  learn  from  the  Western  Apache  about  an  enduring  sense  of  place.  For  the  Western 
Apache,  an  enduring  sense  of  place  comes  from  “lives  spent  sensing  places”  and  a slow  accrual  of  “multiple  lived 
relationships  that  people  maintain  with  places”  (Basso,  1996:  144,  106).  Basso,  therefore,  concludes  that 

this  reciprocal  relationship  - a relationship  ir.  which  individuals  invest  themselves  in  the  landscape  while 
incorporating  its  meanings  into  their  own  fundamental  experience  - is  the  ultimate  source  of  the  rich 
sententious  potential  and  functional  versatility  of  Western  Apache  place-names  (Basso,  1996,  102). 

If  we  arc  going  to  develop  a concept  of  place,  we  arc  also  going  to  have  to  look  at  how  people  live  their 
lives  sensing  places  and  how  they  maintain  lived  relationships  with  other  people  in  specific  places.  We  will,  in 
effect,  have  to  follow  Basso’s  lead  and  undertake  an  “ethnography  of  lived  topographies”  if  we  arc  going  to  make 
any  progress. 

An  enduring  sense  of  place  for  the  Western  Apache  also  entails  a close  identification  of  person,  place,  and 
the  moral  imagination.  Once  this  is  done,  sensing  places  becomes  a common,  everyday  cultural  activity  and  place- 
making  becomes  a “universal  genre  of  experience”  and  a “universal  tool  of  the  human  imagination.”  Basso, 
therefore,  concludes  that 

if  place-making  is  a way  of  constructing  the  past,  a venerable  way  of  doing  history,  it  is  also  a way  of 
constructing  social  traditions  and,  in  the  process,  personal  and  social  identities.  We  are  in  a sense  [under 
such  conditions],  the  place-worlds  we  imagine  (Basso,  1996:  7). 

This  suggests  that  an  ethnography  of  lived  topographies  cannot  simply  deal  with  the  physical 
characteristics  of  places  but  must  also  deal  with  the  moral  imagination  that  persons  bring  to  the  sensing  of  those 
places. 

Finally,  an  enduring  sense  of  place  for  the  Western  Apache  entails  a “complex  array  of  symbolic 
relationships  with  their  physical  surroundings”  (Basso,  1996:  66).  These  symbolic  relationships  arc  not  lived  in 
terms  of  abstract  general. Jes,  but  in  terms  of  “symbolically  drawn  particulars”  (Basso,  1996:  144).  This  suggests 
that  an  ethnography  of  lived  topographies  has  to  combine  ’he  symbolic  and  the  concrete. 

An  Agenda  for  Further  Research  About  Place:  More  Questions 

The  discussion  so  far  has  highlighted  a general  rationale  for  the  importance  of  physical  place  in  our 
thinking  and  elaborated  specific  characteristics  of  a concept  of  place  that  fit  into  this  larger  rationale.  Basso’s 
example,  on  the  other  hand,  has  offered  some  specific  ways  that  physical  place  plays  a role  in  the  daily  practice  of 
another  culture  and  what  that  might  suggest  for  an  ethnography  of  lived  topographies.  What  do  all  these  conclusions 
suggest  for  further  research  in  our  field?  I will  pose  a number  of  questions  which  I believe  emerge  from  the 
discussion. 

The  most  general  rationale  for  the  importance  of  physical  place  in  our  lives  has  to  do  with  reconnecting  to 
the  physical  world  as  a culture  in  the  interests  of  long-term  sustainahi’ity.  David  Orr,  in  Ecological  Literacy , makes 
the  case  that  all  professions  in  the  modern  world  have  to  reexamine  ow  they  are  a contributory  pari  of  modernism 
and  how  they  can  become  a contributory  pari  of  “ecological  postmodernism.”  The  most  general  question  I have, 
therefore,  is: 

1.  how  can  we  reformulate  the  ideas  and  practices  of  our  field  so  that  they  contribute  to  the  transition  to  an 
ecolog ica lly -sustain able  society  ? 

This  question  might  seem  so  general  that  it  is  of  little  practical  value.  However,  if  the  key  to  long-term 
ecological  sustainability  is  an  “uncompromising  commitment  to  life  and  its  preservation”  (Orr,  1992:  133)  and 
one  way  such  a commitment  can  be  pursued  is  by  constantly  reconnecting  our  life-worlds  with  the  physical 
matrix  of  our  existence  no  matter  what  our  culture,  then  we  can  develop  some  operational  specificity  to  the 
general  question  by  focusing  on  physical  place.  The  question  then  becomes: 

2.  how  can  we  reformulate  the  ideas  and  practices  of  our  field  so  that  they  contribute  to  a sense  of  connectedness 
with  our  physical  world? 

I have  briefly  shown,  via  Basso’s  work,  how  the  Western  Apache  culture  has  done  this.  However,  how 
do  we  reformulate  the  ideas  and  practices  of  our  field  when  our  personal  and  social  lives  arc  so  mediated  by 
technology  that  our  distance  from  the  physical  world  may  be  great?  The  answer  to  this  latter  questions  is 
surprisingly  simple:  we  never  left  the  physical  world  in  the  first  place,  we  just  theorized  about  it  as  if  it  did  not 
matter.  Hence,  all  we  have  to  do  is: 
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a.  theoretically  - bring  the  phenomena  of  everyday  practice  back  into  view,  and, 

b.  methodologically  - bring  an  “ethnography  of  lived  topographies”  into  our  research. 

This  will  then  help  us  document  how  people  actually  construct  personal  and  social  meanings  and  then  help  us 
investigate  how  they  develop  a sense  of  connectedness  with  the  material  world. 

For  persons  such  as  Heim  who  begin  by  investigating  the  phenomenological  experience  of  virtual 
reality,  their  research  indicates  a bifurcation  of  the  experience  of  virtual  and  natural  worlds.  The  question  then 
becomes: 

3.  How  can  we  incorporate  the  experience  of  physical  reality  into  the  very  concepts  and  practices  of  our  field? 

My  suggestion  is  that  we  incorporate  a relational  concept  of  place  into  our  thinking  and  practice.  The 
Western  Apache  provide  a very  comprehensive  example  of  a relational  concept  of  place  in  practice  where  an 
understanding  of  placehood  contributes  to  an  understanding  of  selfhood  and  social  survival. 

At  this  point,  a whole  line  of  inquiry  could  branch  off  and  investigate  how  personal  and  social 
identities  are  constructed  on  the  basis  of  body-experience  rather  than  place-experience.  I have  not  pursued  this 
line  of  inquiry.  Others  have,  from  Edmund  Husserl,  who  based  his  phenomenology  on  the  experience  of  the 
body  (Husserl,  I960),  to  Luce  Irigaray,  who  argued  that  all  spaces  are  gendered  (and  contested)  because  they 
arc  ultimately  based  on  the  experience  of  gendered  bodies  (Irigaray,  1993).  However,  I want  to  stay  focused  on 
the  concept  of  physical  place  as  a foundational  concept  because  I want  to  address  the  notion  of  connectedness 
rather  than  difference.  This  means  dealing  with  how  real  people  live  together  in  real  places  with  other  beings. 
The  question  then  becomes: 

4.  how  can  an  ethonography  of  lived  topographies  be  woven  into  the  work  of  our  field  so  that  our  ideas  and 
practices  reflect  a connection  with  the  physical  world? 

I have  deliberately  left  the  issue  of  “digital  living”  open  (i.c.,  when  only  the  technologically-mediated 
representations  of  a person  are  physically  present).  Any  concept  of  physical  place  that  entails  an  accrual 
process  of  living  in  physical  places  would  have  to  accommodate  the  physical  as  well  as  the  semiotic  dimensions 
of  representations.  I know  this  is  an  odd  way  of  staling  the  issue  because  semiotic  theories  deal  more  with  the 
semantic  content  of  representations  than  with  the  physicality  of  representations.  However,  if,  as  the  Western 
Apache  have  shown  in  their  culture,  physical  places  are  “brought  fully  into  being”  through  a process  of  sensing 
and  dwelling  in  storied  places,  then  a question  for  our  field  becomes: 

5.  What  is  brought  fully  into  being  in  our  cultural  context  through  a process  of  sensing  and  dwelling  in  physical 
places  with  mediated  representations  of  people  and  places?  (i.e.,  what  stories  come  along  with 
representations  ? ) 

For  the  time  being,  however.  I only  want  to  focus  on  the  effect  that  a process  of  sensing  and  dwelling 
in  physical  places  has  on  our  construction  of  meaning.  This  line  of  inquiry,  therefore,  leads  to  the  question: 

6.  What  characteristics  would  a relational  concept  of  place  have  to  have  to  begin  to  answer  all  of  the  previous 
questions? 

From  the  earlier  part  of  this  paper,  a relational  concept  of  place  should  bring  us  into  closer  relation 
with  the  physical  world  and  its  patterns  of  “renewing  rhythms.”  This  means  that  it  would  have  to  situate  us  in 
the  physical  realm  of  inhabitation  and  in  the  symbolic  realm  of  our  social  understandings  while  simultaneously 
fostering  a connectedness  with  the  larger  patterns  of  the  world. 

A relational  concept  of  place  should  also  foster  a greater  understanding  of  our  physical  and  social 
placcworld  by  using  terms  which  apply  equally  well  to  person,  place,  and  community.  This  means  using 
terminology  that  avoids  both  reification  of  abstract  concepts  and  reification  of  concrete  places.  Let  me  give  you 
an  example  from  Basso.  Basso  shows  that  the  Western  Apache  sense  of  place  is  not : (Basso,  1996:  143): 

• an  instinctual  need  of  survival  (although  it  contributes  to  surv  ival) 

• a beneficial  attribute  of  healthy  personality  development  (although  it  may  have  that  effect) 

• a mechanism  of  social  integration  (although  it  may  have  this  effect) 

In  my  discussion,  I have  also  argued  that  the  Western  Apache  sense  of  place  is  not: 

• a cognitive  structure  of  visual  memory. 

Each  of  these  terms  (i.c.,  need,  attribute,  mechanism,  and  structure)  arc  abstractions  which  become 
reified  abstractions  when  used  as  explanations.  This  is  something  that  a relational  concept  of  place  seeks  to 
avoid.  On  the  other  hand,  Basso  has  also  shown  that  specific  physical  places  arc  not  absolute,  objective  entities 
which  have  power  independently  of  cultural  context.  Hence,  in  Western  Apache  culture,  specific  places  only 
“speak”  once  a pattern  of  lived  relationships  has  occurred  in  that  place.  This,  in  effect,  avoids  a reification  of 
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specific  places.  I w;i-  generalize  Basso’s  conclusions  by  claiming  that  any  relational  concept  of  place  should 
avoid  these  dual . '^cations  but  rather  express  a particularist  integration  of  the  physical  and  social  worlds. 

A lelauonal  concept  of  place  should  also  help  us  articulate  an  ecology  of  learning.  The  word  ecology 
is  used  rather  loosely  here  but  it  gains  specificity  if  we  follow  Cobb  and  believe  that  this  should  entail  a 
conjoining  of  body  and  place  image.  Let  me  give  you  some  specific  research  questions  that  would  emerge  from 
such  an  approach: 

a.  what  are  the  semiotic  relationships  that  teachers  and  learners  have  with  respect  to  their  physical  places  of 
teaching  and  learning  and  how  does  this  semiotic  relationship  affect  the  effectiveness  of  teaching  and 
learning? 

To  get  a glimpse  of  an  answer,  imagine  in  your  mind,  if  you  will,  the  best  teachers  you  ever  had.  Do 
you  remember  the  place  where  your  interaction  with  them  took  place?  Did  that  place  play  a role  in  your 
interaction? 

b.  what  communal  stories  about  the  places  of  teaching  and  learning  do  people  bring  to  bear  in  the  act  of 
teaching  and  learning  and  how  do  these  stories  shape  their  subsequent  ideas  and  actions? 

c.  how  do  individuals  situate  themselves  in  these  storied  teaching  and  learning  places  and  how  do  they 
appropriate  these  stories  into  their  personal  biographies? 

d.  how  do  individual  appropriations  of  storied  teaching  and  learning  places  reenter  larger  communal 
histories ? 

e.  how  should  we  design  physical  teaching  and  learning  places  so  that  when  people  engage  in  virtual 
teaching  and  learning  at  a distance,  the  fused  physical/virtual  places  still  connect  them  to  sustainable 
patterns  in  the  larger  world? 

A relational  concept  of  place  also  has  to  lead  to  an  understanding  of  a sense  of  place  that  is  both  rooted 
in  the  real  and  connected  to  multiple  communities  of  human  and  more-than-human  beings.  One  way  to  achieve 
this  is  to  take  a phenomenological  approach  and  investigate  how  people  are  engaged  in  a “perceptual 
reciprocity”  with  places  and  other  beings.  A phenomenological  approach  permits  us  to  see: 

a.  how  a place  is  given  its  character  by  the  pattern  of  lived  relationships  that  transpire  there, 

b.  how  a person  builds  up  a biography  on  the  basis  of  perceptually  engaging  specific  places  and  people, 

1 . how  a community  builds  up  semiotic  relationships  with  places  and  other  people. 

In  the  case  of  the  Western  Apache,  Basso  used  a phenomenological  approach  to  show  how  their  daily 
discoursc/practices  fused  person,  place,  and  stories  into  living  semiotic  relationships.  What  would  a 
phenomenological  approach  show  us  about  our  daily  discourse/practices? 

If  the  ideas  presented  so  far  have  any  merit,  then  a relational  concept  of  place  will  have  to  uncover 
specific  cultural  constructions  that  reflect  the  biographies  and  histories  of  lived  relationships.  In  the  case  of  the 
Western  Apache,  their  relational  concept  of  place  was  flexible  enough  to  be  tied  to: 

a.  specific,  concrete  storied  places, 

b.  a specific  and  concrete  historical  community, 

c.  a specific  and  unique  people’s  moral  imagination, 

d.  their  personal  and  communal  identities, 

2.  their  survival  as  an  historical  community  in  a larger  material  world. 

In  our  case,  we  would  have  to  develop  a relational  concept  of  place  where  large  parts  of  our 
discoursc/practices  entail  technological  mediation.  Virtual  reality  technologies  pose  a particularly  difficult 
problem  here  because  they  seem  to  extract  abstract  patterns  from  every  physical  substrate  except  computational 
devices.  Hence,  as  Heim  has  shown,  even  though  computational  representations  arc  physically  embodied  in 
computers,  virtual  worlds  are  controlled  by  computational  abstractions.  Once  these  abstract  representations  are 
linked  by  data  gloves,  head-mounted  displays,  and  even  tactile  sensors  to  our  entire  body,  they  have  the  power 
to  completely  disconnect  our  “perceptual  reciprocity”  with  physical  places.  The  question  then  becomes: 

7.  How  can  we  develop  a relational  concept  of  place  in  light  of  virtual  reality  developments? 

Indirectly,  Basso  again  provides  an  answer.  Basso  showed  how  a “dialogue  with  real  places”  merged 
the  abstract  with  the  concrete  because  the  biography  and  history  of  a person  was  brought  to  bear  on  the  sensing 
of  places.  This  was  a fusion  of  perception  and  imagination.  Likewise,  if  wc  attend  to  the  biography  and  history 
that  a person  brings  to  bear  on  every  new  situation,  whether  real  or  virtual,  and  if  we  attend  to  what  people 
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actually  do  in  technologically-mediated  situations,  then  wc  are  likely  to  avoid  developing  ideas  which  split  the 
abstract  from  the  concrete.  In  education,  this  approach  would  point  us  towards  issues  of  real  learning  whether 
wc  are  talking  about  real  or  virtual  places.  “Real  learning,”  writes  David  Orr  in  Ecological  Literacy , “always 
increases  intelligence”  (Orr,  1996:  xi).  Real  learning,  to  paraphrase  Christopher  Day,  offers  “a  sense  of  roots  ... 
in  the  wider  natural  world  with  its  renewing  rhythms”  (Day,  1990:  26). 

I would  like  to  end  here  with  a tentative  answer  to  my  original  question:  when  is  it  important  to  be 
physically  present  in  teaching  and  learning?  Even  though  I have  not  addressed  this  question  directly,  but  rather 
answered  the  why  question  in  this  paper,  my  answer  is  simply:  always.  We  should  always  be  consciously  present 
where  wc  are  physically  emplaced. 
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A Paradigm  for  Enhancing  Course  Offerings  Using  CD-ROM,  Interactive 

Video  and  E-mail 

Jerry  Summers 
Larry  Reck 

Indiana  State  University 


Introduction 

Competition  for  and  recruitment  of  students  in  higher  education  have  spawned  the  development  of  distance 
education  programs.  Institutions,  more  than  ever,  arc  seeking  ways  to  attract  learners  into  time-  and  cost-effective 
programs.  Multiple  delivery  mechanisms  have  subsequently  evolved:  interactive  television,  computer  based 
programs  via  CD-ROM,  lascrdisks,  floppy  disks,  Internet  and  computer  based  compressed  video.  Selection  of  the 
delivery  mechanism  is  generally  the  instructor’s  choice.  Interactive  television  and  computer  based  video  generally 
involve  synchronous  communications  and  interactions.  Instructional  techniques  tend  to  be  “live,”  more  formalized 
and  teacher  directed. 

Other  modalities  arc  asynchronous.  In  asynchronous  communications,  the  instructor  and  the  learners 
interact,  but  not  necessarily  in  ‘'real  time.”  The  learner  subsequently  has  complete  choice  with  respect  to  when  he  or 
she  desires  to  study.  Asynchronous  delivery  mechanisms  may  involve  CD-ROM,  laserdisks,  interactive  video 
floppy  disks,  Internet  or  other  computer  based,  non-real  time  technologies. 

If  instruction  can  be  provided  at  times  that  are  convenient  to  students,  then  larger  enrollments  may  be 
predicted.  However,  the  immediate  inaccessibility  of  the  instructor  provided  impetus  for  charges  relative  to  lack  of 
quality,  Hollywood  hype,  lack  of  control,  and  mass  production  of  learning. 

Purpose 

The  purpose  of  this  study  is  to  describe  the  results  of  a year-long  effort  to  explore  asynchronous  delivery 
modalities  in  a distance  education  program.  The  study  investigates  a paradigm  for  enhancing  interactivity  in 
distance  education  using  CD-ROM,  interactive  video  and  e-mail.  The  purposes  of  the  investigation  were  to  (a) 
create  an  asynchronous  CD-ROM,  interactive  video  and  e-mail  based  distance  learning  environment,  (b)  build  into 
the  program  multiple  interactive  prompts  (readiness,  conditional,  non-linear  branching,  consequential,  and 
reflective)  and  delivery  techniques  (interactive  video,  textual,  and  graphical),  and  (c)  evaluate  both  accessibility  and 
program  impact  upon  student  learning  and  attitudes.  As  the  sophistication  of  distance  education  technologies 
develops,  program  designers  will  be  required  to  evaluate  their  products  and  efforts  in  ways  not  unlike  how 
instruction  is  evaluated  in  classrooms  all  over  the  globe.  This  work  is  directed  at  extending  this  conversation. 

Discussion  on  Interactivity 

This  study  is  a continuation  of  previous  exploration  on  computer-based  interactivity  by  the  authors.  A 
subsequent  study  examined  the  effect  of  interactivity  on  student  achievement  and  affective  perceptions  and 
discovered  that  when  delivery  modality  was  controlled,  no  differences  in  pupil  achievement  or  affective  perceptions 
were  obtained.  This  finding  was  judged  “curious”  and  potentially  flawed,  however  subsequent  investigations 
reinforced  the  finding.  One  hypothesis  for  the  findings  suggested  that  not  all  prompts  are  equal;  some  interactive 
approaches  may  be  more  effective  than  others.  This  finding  eventually  led  to  a paradigm  for  computer-based 
interactivity  that  involved  four  types  of  prompts  that  were  defined  as  follows: 

Readiness  Prompt  - a prompt  in  which  the  user  simply  decides  whether  or  not  to  continue  in  the  program. 

Conditional  Prompt  - a prompt  in  which  continuation  in  the  program  is  contingent  upon  successful 
completion  of  previous  assignments. 

Non-Linear  Prompts  - a prompt  in  which  the  user  simply  chooses  from  among  multiple  alternatives  which 
topic  he  or  she  desires  to  pursue. 

Consequential  Choice  - a prompt  for  which  the  selection  has  consequences  (the  patient  dies,  the  bow  runs 
out  of  arrows,  etc.) 

Reflective  - a prompt  involving  written  reflection  or  thinking  about  a topic  or  event. 
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The  expectation  of  the  researchers  was  that  programs  involving  these  prompts  would  entail  different 
learning  and  different  attitudes  on  the  part  of  students. 


Overview 

A combination  of  computer  programs,  CD-ROM,  videotapes  and  e-mail  served  as  the  delivery  mechanism 
for  this  graduate  level  Secondary  School  Curriculum  course.  During  the  only  meeting  on  campus,  students  were 
introduced  to  a new  method  of  instruction  at  Indiana  State  University.  Logistics  and  methodology  were  discussed 
and  students  were  given  a computer  program  consisting  of  both  the  course  content  and  instructional  methodology. 
Briefly,  content  consisted  of  reading,  responding,  and  interacting  with  (1)  Goals  2000,  (2)  textbooks,  (3)  projects 
selected  from  over  fifty  current  trends  in  the  education  field,  and  (4)  responding  to  social  behaviorisl,  cxpericntialisl 
and  traditional  philosophies  viewed  via  interactive  video  with  “attacks”  disseminated  to  other  class  members  on  a 
distribution  list,  (5)  other  course  requirements.  Fifteen  assignments  in  all  were  required.  Responses  were  c-mailed 
to  the  instructor  for  evaluation  and  comments.  For  one  out  of  twenty-five  students  not  having  e-mail,  content  was 
submitted  via  floppy  disk.  Distribution  lists  were  sent  to  the  class  members  to  react  with  one  another  on  their 
various  stands  on  law  cases,  etc. 

Instructor  Goals  For  Participants 

1.  To  enable  students  to  become  proficient  in  curriculum  planning,  its  foundations,  types,  forces,  processes, 
evaluation,  criticism,  latest  trends,  decision  making  and  future  direction. 

2.  To  enroll  students  who  would  not  otherwise  partake  of  course  offerings  by  improving  accessibility;  c.g.,  non- 
traditional  students. 

3.  To  apply  curriculum  content,  its  design,  implementation  and  evaluation  into  the  student’s  own  teaching-learning 
situation  and  to  become  aware  of  curricular  issues  and  criticism  and  be  able  to  formulate  a response  to  them. 

4.  To  provide  quality  instruction  through  the  latest  technologies. 

5.  To  improve  cost  effectiveness  by  reaching  more  students  at  a lower  expenditure. 

6.  To  impact  students  with  the  application  and  direct  utilization  of  computers,  networking  and  e-mail  to  conduct 
learning  activities. 

7.  To  acquaint  students  with  the  role  of  technology  in  the  curriculum  and  the  integration  of  media  into  the  teaching- 
learning  process. 

8.  To  allow  flexibility  of  time  management  by  studying  at  home  in  student’s  controlled  environment. 

9.  To  enable  students  to  become  more  responsible  as  an  active  participant  in  curriculum  development  by  becoming 
knowledgeable  of  the  patterns  and  processes  of  effective  curricular  change. 

Delivery  Mechanisms 

CD-ROM  and  interactive  video  provide  considerable  capability  for  asynchronous  course  delivery.  This 
program  involved  instruction  that  was  also  based  upon  graphic,  and  textual  materials.  Learners  were  asked  to 
receive  instruction  in  the  prescribed  way  and  respond  via  e-mail  to  other  class  members  and  the  instructor.  CD- 
ROM  was  used  to  take  advantage  of  the  power  and  speed  of  software  independent  of  Internet;  also,  it  is  a medium 
for  which  we  have  property  rights  and  will  not  be  widely  distributed. 


Results 

Of  the  four  projects  required  as  of  this  date  (six  weeks  into  the  semester),  completions  stand  as  follows  for 
the  25  students  enrolled: 

Project  1:  1 incomplete  - 96%  completion  rate 
Project  2:  0 incomplete  - 100%  completion  rate 
Project  3:  4 incomplete  - 84%>  completion  rate 
Project  4:  5 incomplete  - 80%  completion  rate 

A few  people  have  completed  projects  ahead  of  the  due  dates  including  one  person  who  has  only  one  assignment 
remaining  for  completion  of  the  course.  He  should  have  all  requirements  in  before  the  mid-point  of  the  semester. 


Analysis  of  Results  and  Problems 

Due  to  various  computer  problems,  not  all  students  were  online  at  the  beginning  of  the  course.  Although 
they  could  use  the  post  office  to  mail  their  projects  to  the  instructor  until  they  obtained  the  necessary  hardware, 
some  did  not  take  the  initiative  to  do  that.  Hence  a major  problem  of  this  type  of  distribution  mechanism  is  evident 
in  that  it  requires  personal  motivation  to  complete  the  course.  Those  who  do  not  have  this  attitude  and  desire 
probably  are  not  life-long  learners  anyway  and  would  fail  in  other  similar  attempts,  hut  for  most  students  in  the  past 
two  years  this  has  been  a very  successful  venture,  for  the  accolades  and  phrases  have  been  rewarding  for  the  time 
and  energy  pul  into  this  type  of  offering.  Of  those  responding  in  the  past  to  the  course  evaluation,  the  ratings  were: 
A,  64  students;  B+,  5 students;  B,  5 students.  There  were  no  lower  grades  given  to  this  course,  therefore  it  is  deemed 
a successful  venture  and  will  continue  to  be  offered  in  the  future,  of  course  with  up-grades  to  keep  the  content  and 
methodology  current. 
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The  Role  of  Awareness  of  Cognitive  Style  in  Hypermedia 


Jennifer  B.  Summerville 
Emporia  State  University 


Abstract 

The  focus  of  this  dissertation  was  to  examine  the  variables  of  cognitive  style,  subject  awareness  of  the 
instructional  implications  of  cognitive  style  and  niatching/mismatching  subjects  with  cognitive  style.  These 
variables  may  be  important  in  the  design  of  instructional  environments,  such  as  hypermedia,  adapted  to 
accommodate  individual  differences. 


Introduction 

Individual  differences  arc  becoming  a focus  of  current  instructional  design  and  practice.  It  is  unrealistic, 
however,  to  expect  teachers  in  all  educational  settings  to  alter  educational  environments  in  order  to  meet  each 
student’s  educational  needs  such  as  differences  in  cognitive  style.  Some  of  the  responsibility  for  learning  must  rest 
with  the  learner. 

Currently,  many  students  are  unaware  of  how  knowledge  of  their  own  individual  differences,  such  as  field 
d^pcndcnce/indcpendence,  affect  the  ways  in  which  they  learn  (Jones,  1993).  The  focus  of  this  study  is  to  examine 
some  of  the  critical  variables  which  may  be  important  in  the  design  of  instructional  environments  adapted  to 
accommodate  individual  differences. 

Matching  cognitive  style  to  teaching  environments  may  be  important  because  of  the  potential  to  enhance 
learning.  However,  at  this  time,  the  relationship  between  matching  cognitive  styles  and  learning  has  not  been 
researched  fully  and  the  implications  are  inconclusive,  especially  for  hypermedia  learning  environments. 

This  researcher  hypothesized  that  hypermedia  could  be  designed  to  capitalize  on  a student’s  tendency  for 
field  dependence  or  field  independence  and  also  designed  to  compensate  for  any  learning  difficulties  field 
dcpcndenl/independent  students  may  encounter  due  to  their  tendency.  In  the  past,  researchers  have  concentrated  on 
variables  other  than  those  directly  related  to  field  dependencc/indepcndencc  which  may  have  confounded  their 
results. 

Based  on  the  review  of  the  literature,  this  researcher  believed  that  one  of  the  main  problems  with  studies 
that  match  and  mismatch  subjects  with  their  own  cognitive  style  was  that  the  variable  of  awareness  of  cognitive 
style  had  not  been  taken  into  consideration.  Not  including  this  variable  presented  some  potential  problems.  For 
example,  the  subject  may  be  aware  of  his  or  her  ‘‘style”  preference  prior  to  the  treatment,  especially  in  the  case  of 
learning  styles  and  environmental  preferences.  As  a result,  mismatching  the  subjects  to  learning  styles  other  than 
their  own  could  possibly  alter  learning  outcomes.  In  contrast,  the  subject  may  be  unaware  of  his  or  her  “style” 
preference  and  the  potential  importance  of  this  preference  prior  to  the  treatment,  again  altering  learning  outcomes. 

Purpose  of  the  Study 

There  were  several  purposes  for  this  study.  The  first  was  to  determine  whether  matching  or  mismatching 
subjects  with  their  tendency  toward  field  dependence  or  field  independence  had  any  effect  on  achievement  in  a 
hypermedia  learning  environment.  A second  purpose,  related  to  the  first,  was  to  determine  whether  matching  or 
mismatching  subjects  with  their  tendency  toward  field  dependence  or  field  independence  had  any  effect  on 
satisfaction  in  a hypermedia  learning  environment.  The  third  purpose  was  to  examine  the  role  of  awareness  of  field 
dcpendencc/independence  as  students  learned  in  a hypermedia  environment  and  the  resulting  effect  on  achievement. 
The  fourth  was  to  examine  the  role  of  awareness  of  field  dcpendcnce/independcnce  as  students  worked  in  a 
hypermedia  environment  and  the  effect  on  satisfaction.  The  final  purpose  was  to  explore  possible  interactions  of  the 
variables:  awareness  of  cognitive  style,  field  dcpcndcnce/indcpcndcncc,  and  match/mismatch  with  cognitive  style  in 
a hypermedia  environment. 
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Statement  of  the  Research  Questions 

The  following  research  questions  and  hypotheses  guided  this  study. 

1.  To  what  extent  does  matching  a student  with  a hypermedia  environment  designed  with  instructional 
support  for  field  dependent  or  field  independent  cognitive  style  affect  student  achievement  in  learning 
HyperCard? 

2.  To  what  extent  does  mismatching  a student  with  a hypermedia  environment  designed  with  instructional 
support  for  field  dependent  or  field  independent  cognitive  style  affect  student  achievement  in  learning 
HyperCard? 

3.  To  what  extent  does  matching  a student  with  a hypermedia  environment  designed  with  instructional 
support  for  field  dependent  or  field  independent  cognitive  style  affect  student  satisfaction  with  the 
HyperCard  lesson  and  the  learning  experience? 

4.  To  what  extent  does  mismatching  a student  with  a hypermedia  environment  designed  with  instructional 
support  for  field  dependent  or  field  independent  cognitive  style  affect  student  satisfaction  with  the 
HyperCard  lesson  and  the  learning  experience? 

5.  To  what  extent  docs  awareness  of  field  dcpcndcnce/independencc  affect  student  achievement  in  a 
hypermedia  HyperCard  lesson? 

6.  To  what  extent  does  awareness  of  field  dcpendencc/indepcndcnce  affect  student  satisfaction  with  a 
hypermedia  HyperCard  lesson? 

7.  To  what  extent  do  these  three  variables  (learner  awareness  of  field  dependcncc/indcpendcnce, 
matching/mismatching  HyperCard  lessons,  field  dcpendencc/indepcndcnce),  singly  or  in  combination, 
affect  achievement  in  learning  HyperCard? 

8.  To  what  extent  do  these  three  variables  (learner  awareness  of  field  dcpendencc/indepcndcnce, 
matching/mismatching  HyperCard  lessons,  field  dcpcndencc/indcpcndence),  singly  or  in  combination, 
affect  satisfaction  with  the  HyperCard  lesson  and  the  learning  experience? 

Significance  of  the  Study 

Kogan  (1971)  proposes  that,  “Wilkin’s  field  dcpendcncc/indcpcndencc  dimension  would  appear  to  he 
ideally  suited  for  research  on  the  interaction  between  variables  of  cognitive  style  and  instructional  treatment.  Both 
ends  of  Wilkin’s  dimension  have  adaptive  properties,  though  of  a distinctly  different  kind,  and  it  is  feasible  that 
educational  programs  could  be  devised  to  profit  each  of  the  polar  types”  (p.  252). 

The  combination  of  identifying  cognitive  style,  instructional  treatment  supported  by  design  features  related 
to  the  cognitive  style  characteristics  and  learner  awareness  of  cognitive  style  may  prove  to  have  significant  impact 
on  student  achievement  on  a given  task  and/or  student  satisfaction  with  that  task. 

This  researcher  examined  variables  identified  as  important  for  future  research  in  other  studies. 
Specifically:  (a)  determining  whether  or  not  field  dependencc/independencc  can,  or  should,  be  embedded  within  a 
hypermedia  environment;  (b)  discovering  what  role  awareness  of  cognitive  style  plays  in  achievement  and 
satisfaction  with  a hypermedia  learning  environment;  and  (e)  defining  which  interactions  between  variables  hold  the 
most  potential  for  increasing  achievement  and  satisfaction. 

Definition  of  Terms 

The  following  terms  with  the  stated  definitions  were  used  extensively  throughout  this  study. 

Awareness  of  Cognitive  Style  - The  degree  to  which  an  individual  understands  the  instructional 
implications  of  specific  cognitive  style  characteristics. 

Field  Pependenee/Indcpcndencc  - The  degree  to  which  an  individual's  processing  of  information  is 
effected  by  the  contextual  field. 
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Matching  Instructional  Environments  - Instructional  environments  designed  to  match  a student’s  cognitive 
style— in  this  case,  Held  dcpendencc/indcpendcncc. 

Mismatching  Instructional  Environments  - Instructional  environments  designed  to  be  mismatched  with  a 
student’s  cognitive  style-in  this  case,  field  dcpendence/indcpendence. 

Satisfaction  - The  degree  to  which  students  are  satisfied  with  the  learning  environment,  arc  comfortable 
with  the  learning  environment,  and  perceived  that  they  learned  from  the  experience. 

Literature  Review 

Field  Dependence/Independence 

This  researcher  chose  to  begin  with  the  most  comprehensive  literature  review  of  field 
dependcnce/indepcndcnce  (Witkin  et  ah,  1977)  and  the  Manual  for  the  Embedded  Figures  Tests  (Witkin,  Oilman, 
Raskin,  & Karp,  1971)  because  of  their  extreme  importance  in  setting  both  a theoretical  and  practical  foundation. 
Witkin  and  colleagues  (1977)  detailed  the  research  done  in  the  area  of  field  depcndence/independence  over  twenty- 
five  years.  Field  dependence/independenee  appears  to  affect  many  aspects  of  daily  life  including  the  ability  to  learn 
from  social  environments,  types  of  educational  reinforcement  needed  to  enhance  learning,  amount  of  structure 
preferred  in  an  educational  environment,  cue  salience,  interactions  between  teachers  and  students,  and  career 
choices. 

However,  despite  the  plethora  of  n ' ' detailed  by  Witkin  and  his  colleagues  (1977),  these  researchers 

no»  additional  areas  for  further  research.  an  f the  suggestions  were  in  the  area  of  interactions  between  field 
u [.  dence/independence  and  the  charactci..mcs  f the  learning  environment.  It  is  unknown  whether  some  of  the 
behaviors  exhibited  by  teachers  during  research  experiments  arc  due  to  the  characteristics  of  field 
depcndence/independence  or  whether  these  behaviors  arc  in  fact  due  to  the  Hawthorne  Effect,  an  observed  change  in 
research  participants’  behavior  based  on  their  awareness  of  participating  in  an  experiment  (Borg,  Gall,  & Gall, 
1996). 

Another  question  raised  by  Witkin  cl  al.  was  whether  teachers  can  adapt  their  teaching  techniques  to 
accommodate  students  with  different  cognitive  styles.  Related  to  this  question  is,  what  is  the  effect  for  both  the 
teachers  and  the  students  if  the  teacher  is  sensitized  to  the  inherent  differences  for  field  dependent  and  independent 
students? 

Several  pages  of  the  Witkin  et  al.  (1977)  article  were  devoted  to  the  conflicting  research  in  the  area  of 
matching/mismatching  with  cognitive  style.  The  researchers  only  cornmitcd  to  the  fact  that  matching/mismatching 
with  cognitive  style  was  a factor  in  teacher-student  interaction.  An  unknown  variable  was  whether  match/mismatch 
actually  improved  learning  and,  if  so,  how  it  improved  learning.  The  researchers  also  noted  that  other  situational 
variables  (i.c.,  the  study  moderator)  may  have  impacted  the  results  on  matching/mismatching. 

Jonassen  and  Grabowski  (1993)  devoted  an  entire  chapter  of  The  Handbook  of  Individual  Differences  to 
the  cognitive  style  of  field  dependence  and  field  independence  (a.k.a.  global  vs.  articulated  style).  They  summarized 
the  research  on  the  implications  of  the  style  characteristics  as 

Instructional  conditions  that  capitalize  on  the  preferences  of  the  field  dependent  student  and 

challenge  the  field  independent  student  include: 

1 . providing  a syncrgogic  (social)  learning  environment; 

2.  offering  deliberate  structural  support  with  salient  cues,  especially  organizational  cues 
such  as  advanced  organizers; 

3.  providing  clear,  explicit  directions  and  the  maximum  amount  of  guidance; 

4.  including  orienting  strategics  before  instruction; 

5.  providing  extensive  feedback  (especially  informative); 

6.  presenting  advance  organizers  (verbal,  oral,  or  pictorial); 

7.  presenting  outlines  or  graphic  organizers  of  content; 

8.  providing  prototypic  examples; 

9.  advising  learner  of  instructional  support  needed  (examples,  practice  items,  tools, 
resources); 

1 0.  providing  graphic,  oral  or  auditory  cues; 

1 1 . embedding  questions  throughout  learning;  and 

1 2.  providing  deductive  or  procedural  instructional  sequences  (p.  97). 
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Instructional  conditions  that  capitalize  on  the  preferences  of  the  field  independent  student  and 

challenge  the  field  dependent  student  include 

1 . providing  ar.  jidependent  learning  environment; 

2.  utilizing  inquiry  and  discovery'  teaching  methods; 

3.  providing  abundant  content  resources  and  reference  material  to  sort  through; 

4.  providing  independent,  contract-based  self-instruction; 

5.  providing  minimal  guidance  and  direction; 

6.  asking  the  learner  to  pose  questions  to  be  answered; 

7.  using  inductive  instructional  sequence; 

8.  creating  outlines,  pattern  notes,  concept  maps,  etc.;  and 

9.  using  theoretical  elaboration  sequences  (pp.  97-98). 

These  characteristics  were  used  as  guidelines  for  the  development  of  the  treatment  used  in  this  research. 

Awareness  of  Cognitive  Style 

A thorough  review  of  the  literature  yielded  no  empirical  evidence  of  the  importance  of  awareness  of 
cognitive  style  in  the  design  of  instruction.  However,  several  researchers  (Jones,  1993;  O'Brien,  1989;  Perry,  1994; 
Schmcck,  1988;  Turner,  1993)  suggested  considering  the  importance  of  student  awareness  of  their  own  cognitive 
style  and  what  it  means  educationally.  To  date,  it  is  merely  a suggestion  of  value  and  has  not  been  studied  as  a 
factor  in  research  where  any  cognitive  style,  including  field  dcpcndcnce/indcpcndence  was  a variable.  These 
suggestions  have  an  overall  thcmc-that  awareness  of  cognitive  style  may  increase  the  ability  to  recognize  that 
individual  needs  are  not  being  met.  Consequently,  these  subjects  may  want  to  take  a substantial  part  in  their  own 
learning. 

Schmeck  (1988)  slated  that  “placing  limits  upon  self-knowledge  ...  may  be  what  places  limits  upon  overall 
cognitive  integration.  If  this  is  so,  then  increasing  self-acceptance  will  permit  greater  self-awareness  and  lead 
ultimately  to  a cognitive  style  characterized  by  greater  versatility,  flexibility,  and  adaptation  in  overall  functioning” 
(pp.  149-150).  In  a situation  where  accommodation  of  cognitive  style  is  not  being  considered,  awareness  of 
individual  style  could  lead  to  self-adaptation  to  the  environment  or  to  the  individual’s  request  that  the  environment 
be  altered  to  meet  his  or  her  needs.  This  may  be  critical  in  the  ease  of  the  field  dependent  subject  who  needs  a more 
structured  environment  yet  finds  himself  or  herself  in  a very  unstructured  learning  environment. 

Perry  (1994)  commented  that  if  there  was  an  understanding  of  cognitive/learning  styles,  cooperative 
arrangements  could  be  made  between  individuals  of  different  styles  to  possibly  compensate  for  the  deficiencies  of 
one  style.  Perry  also  suggested  that  an  appreciation  for  diversity  was  important  in  any  educational  environment. 
This  extends  beyond  the  diversity  of  race  and  gender  to  other  ways  of  thinking  and  learning.  According  to  Perry, 
when  we  allow  learners  to  understand  how  they  learn,  there  is  greater  possibility  lor  efficient  and  effective  learning 
and  teaching. 

Jones  (1993)  devoted  an  entire  article  to  the  question,  “Cognitive  learning  styles:  Docs  awareness  help?” 
Jones  suggested  that  cognitive  styles  had  general  applicability,  lended  themselves  to  cross-application  over  various 
subject  matters;  and  had  a broad  usefulness  which  extended  to  other  areas  outside  of  education.  “Due  to  the  fact  that 
the  students  learn  something  about  their  own  mental  processes,  they  may  then  be  able  to  structure  and  make  sense  of 
otherwise- unordered  experience  and  of  their  intuitive  or  random  use  of  procedures.  This  has  great  value  in 
education,  most  particularly  in  self-study  or  independent  learning”  (p.  197). 

Jones  (1993)  suggested  that  once  a learner  is  aware  of  the  characteristics  for  one  type  of  cognitive  style,  the 
learner  may  be  able  to  adopt  learning  strategics  that  will  make  use  of  their  strengths  and  compensate  for  weaknesses. 
Finally,  Jones  concluded  that  it  is  not  enough  to  make  a student  aware  of  these  cognitive  style  characteristics;  it  is 
also  important  to  include  complementing  instructional  strategies  which  take  into  account  these  characteristics. 

Matching  and  Mismatching  with  Cognitive  Style 

Currently,  little  is  known  about  the  exact  relationship  between  matching  or  mismatching  students  with  their 
cognitive  style  and  the  cognitive  style  itself.  Witkin  et  al.  (1977)  suggested  that  these  qu'  ns  be  answered  in 
future  research:  (a)  whether  matching  for  cognitive  style  makes  for  better  student  learning  or  is  it  simply  that 
teachers  and  students  with  the  same  cognitive  style  like  each  other  better?  (b)  if  matching  docs  make  for  better 
student  learning,  why  does  this  occur?  and  (c)  what  other  variables  enter  into  the  maich/mismatch  environment  that 
may  alter  outcomes  of  such  experiments? 
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Shipman  and  Shipman  (1985)  questioned  whether  researchers  currently  know  enough  about  the 
match/mismatch  variable  to  make  exact  determinations  of  instructional  implications.  Since  the  exact  relationship  is 
unknown,  it  is  impossible  to  know  whether  cither  matched  or  mismatched  results  occur  due  to  characteristics  of 
cognitive  style,  interpersonal  incompatibilities  of  students  and  teachers,  inappropriate  teaching  techniques,  or 
differences  in  personal  interests  which  may  be  related  to  the  traits  of  that  cognitive  style.  Rather,  Shipman  and 
Shipman  suggested  that  future  researchers  should  either  focus  on  the  effects  of  the  match/mismatch  or  on  how  to 
create  environments  that  are  adaptive  to  the  needs  of  field  dependent/independent  subjects. 

Hypermedia  and  Cognitive  Style 

Hypermedia  is  a computer  mediated  environment  which  holds  great  promise  for  the  accommodation  of 
individual  differences.  Inherent  in  its  special  properties,  hypermedia  has  the  ability  to  be  flexible  or  structured, 
provide  varied  feedback,  and  allow  the  user  to  access  other  resources  at  the  click  of  a button.  Ayersman  and  Minden 
(1995)  acknowledged  that  there  is  little,  research  regarding  the  relationship  between  hypermedia,  learning,  and 
accommodating  individual  differences.  However,  they  suggested  that  hypermedia  is  the  ideal  way  to  accommodate 
a variety  of  individual  differences,  including  cognitive  style.  The  researchers  stated  that  while  other  forms  of 
traditional  computer-aided  instruction  arc  available,  other  forms  require  that  the  instruction  be  modified  or  that  the 
learner  adapt  to  the  instruction.  Hypermedia,  on  the  other  hand,  has  “the  ability  to  deliver  information  in 
contextually  meaningful  sequences,  at  a variable  pace  controlled  by  the  learner,  and  through  multiple  sensory 
modalities”  (p.  387).  In  other  words,  hypermedia  can  be  developed  to  accommodate  various  learner  needs. 

Little  research  exists  in  the  combined  area  of  hypermedia,  field  dependence/  independence  and  the 
match/mismatch  of  cognitive  style  with  instruction.  However,  two  studies,  both  of  which  examine  matching  aspects 
of  instruction  which  correlate  to  individual  characteristics  of  cognitive  style  in  computer  aided/hypermedia  aided 
instruction,  will  be  discussed. 

Weller,  Repman,  Lan,  and  Rooze  (1995)  conducted  two  studies  which  investigated:  (a)  possible 
mismatches  between  learners’  cognitive  styles  (field  dcpendence/independence)  and  the  presence  of  advance 
organizers  and  structural  organizers  in  the  hypermedia,  and  (b)  possible  mismatches  between  ’earner’s  cognitive 
style  (field  dependence/field  independence),  social  context  of  learning,  and  self-regulation  of  time  during  HBI 
(Hypermedia  Based  Instruction).  Junior  high  subjects  in  both  experiments  used  a computer  ethics  HyperCard  stack 
and  answered  questions  on  a posttest.  Both  studies  concluded  that  field  independent  learners  had  higher 
achievement  scores  overall  but  that  differences  in  treatment  options  (the  presence  or  absence  of  advanced 
organizers)  impacted  achievement  for  field  dependent  subjects  but  not  for  field  independent  subjects.  “These 
findings  suggest  that  designers  of  HBI  [Hypermedia  Based  Instruction]  should  look  for  ways  to  take  into  account  the 
users’  cognitive  styles.  Perhaps  hypermedia  software  should  initially  diagnose  a user’s  cognitive  style  and  then 
provide  optional  modes  of  interactivity  depending  on  this  learner  characteristic”  (Weller  et  al.,  1 995,  p.  463). 

A study  conducted  by  Hedberg  and  McNamara  (1985)  examined  the  relationship  between  type  of  feedback 
and  cognitive  style.  The  researchers  found: 

1.  Field  dependent  subjects  had  longer  first  response  time,  higher  number  of  first  response  errors  and 
higher  total  errors  on  the  task; 

2.  Field  dependent  learners  performed  better  when  given  feedback  containing  an  explanation  of 
errors  and  strategies  for  correcting  errors,  than  when  given  only  an  indication  than  an  error  had 
been  made; 

3.  Field  dependent  students  reduced  their  response  time  and  number  of  errors  when  given  an 
explanation  of  their  errors  and  strategics  for  solving  the  problems;  and 

4.  Providing  detailed  explanations  of  errors  and  strategies  for  problem  solving  did  not  improve 
performance  of  field  independent  learners.  Field  independent  learners  took  less  time  and  made 
fewer  errors  when  g;ven  only  an  indication  that  an  error  had  been  made  (“That’s  not  right. ..Try 
again”). 

Research  Design  and  Methodology 

Subjects 

Since  it  was  csm  aial  that  the  subjects  have  no  prior  knowledge  of  HyperCard,  subjects  currently  enrolled  in 
undergraduate  technology  courses  were  asked  to  volunteer  for  the  study.  Volunteers  from  ET  301  (Educational 
Technology  Applications)  were  given  extra  credit  to  participate  in  the  study  and  could  choose  to  participate  in  lieu 
of  attending  class  to  learn  HyperCard.  Volunteers  from  ET  201  (Technology  in  Education)  could  choose  to  waive 
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cither  an  assignment  involving  questions  from  the  textbook  or  an  assignment  involving  drawing  packages,  in 
addition  to  receiving  extra  credit. 

Internal  Review  Board  approval  was  given  before  conducting  this  study.  Volunteers  were  required  to  sign 
a participant  consent  form  prior  to  the  treatment.  After  the  treatment,  if  the  subjects  did  not  believe  that  they  were 
successful  in  learning  HyperCard,  make-up  sessions  were  offered.  However,  only  one  subject  inquired  about  make- 
up sessions.  This  subject  never  followed  up  with  a requested  for  a specific  time. 

Instrumentation 

Group  Embedded  Figures  Test  (GEFT) 

In  order  to  determine  whether  students  were  field  dependent  or  field  independent,  subjects  were  given  the 
Group  Embedded  Figures  Test  (GEFT)  during  regular  class  time  and  were  assigned  a student  number  based  on  their 
social  security  number.  The  GEFT  is  an  adaptation  of  the  original  Embedded  Figures  Test  (EFT,  developed  in 
1950)  which  has  been  used  to  determine  field  dcpendence/independence  for  large  groups  of  individuals  (Witkin,  H. 
A.,  Oilman,  P.  K.,  Raskin,  E.,  & Karp,  S.  A.  (1971).  This  thirty-two  item  instrument  has  a correlation  of  -.82  for 
male  undergraduates  and  -.63  for  female  undergraduates  with  the  EFT  (Witkin  et  al.,  1971) . The  Spearman-Brown 
reliability  estimate  is  .82  (Witkin  et  al.,  1971).  The  researcher  hand  scored  the  GEFT  using  a template  provided  by 
the  publisher. 

The  HyperCard  Stacks 

Two  HyperCard  stacks  were  developed  by  this  researcher  to  teach  HyperCard,  a requirement  of  the  second 
undergraduate  educational  technology  course.  The  stacks  were  designed  to  assess  the  subject’s  knowledge  of 
HyperCard,  and  to  accommodate  the  special  needs  of  field  independent  and  Field  dependent  subjects.  The  stacks 
were  based  on  the  HyperCard  portion  of  the  instructional  packet  already  in  use  with  current  classes.  This  portion  of 
the  packet  has  been  a part  of  the  undergraduate  educational  technology  courses  since  1995  and  has  been  updated 
regularly  to  correct  errors  and  reflect  changes  in  the  curriculum  and  the  software. 

The  hypermedia  stacks  were  developed  with  careful  consideration  of  the  reported  research  in  the  area  of 
field  dependencc/indcpendencc  while  still  focused  on  the  learning  objectives.  The  accommodations  for  field 
dcpendence/independence  were  mostly  in  the  area  of  instructional  support,  the  structure  of  the  instructional 
environment  and  feedback.  The  instructional  environment  for  field  independence  had  minimal  structure-a  menu 
from  which  subjects  could  choose  to  proceed  through  the  stack  in  any  order.  Instructional  support  was  also  minimal 
and  was  provided  only  when  the  subjects  sought  support  (e.g.,  clickable  text,  scripting  help).  Also,  the  field 
independent  stack  provided  minimal  feedback,  only  telling  the  student  that  he  or  she  was  incorrect  as  they  proceeded 
through  the  learning  task. 

The  field  dependent  stack  was  exactly  the  opposite.  The  field  dependent  stack  was  highly  structured; 
subjects  were  required  to  complete  the  stack  in  sequential  order  before  they  were  allowed  to  go  to  the  next  area. 
Extensive  instructional  support  was  provided;  for  example,  definitions  were  repeated  and  users  were  informed  of 
access  to  both  this  researcher  and  computer  lab  consultants.  Feedback  was  explicit  and  corrective 

Other  than  these  specific  accommodations,  it  is  important  to  emphasize  that  the  instructional  content  itself 
was  identical  for  both  the  field  dependent  and  independent  stacks.  This  ensured  that  both  groups  received  equal 
treatments.  Also,  both  groups  were  allowed  to  take  notes  as  notetaking  was  shown  to  be  of  benefit  to  both  field 
dependent  and  field  independent  subjects  (Frank,  1984).  As  stated  earlier,  both  stacks  were  designed  to  meet  the 
instructional  objectives  detailed  in  the  instructional  packet  currently  in  use  in  undergraduate  educational  technology 
classes. 

Before  administration,  the  treatment  stacks  were  evaluated  by  subject  matter  experts  in  three  areas: 
HyperCard,  field  dcpendence/independence,  and  Educational  Technology  instruction  to  establish  face  and  content 
validity.  The  subject  matter  experts  examined  the  HyperCard  stacks,  checked  the  accuracy  of  the  design  both  for 
content  and  accommodation  of  field  dcpendence/independence,  evaluated  the  assessment  criterion,  and 
recommended  revisions  on  a printed  copy  of  the  stacks.  Based  upon  these  comments,  revisions  to  the  stacks  were 
completed.  This  process  was  repeated  until  the  stacks  were  accurate  and  complete. 


Achievement  Rubric 

The  next  part  of  the  treatment  involved  application  of  the  student  knowledge.  Subjects  were  required  to 
complete  their  own  stacks.  They  had  one  hour  to  complete  a short  stack  which  included  all  of  the  HyperCard 
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elements  that  they  learned  about  during  the  treatment.  They  were  allowed  to  use  the  notes  that  they  took  during  the 
first  portion  of  the  treatment;  no  other  support  was  provided.  These  stacks  were  graded  for  accuracy  according  to  a 
predetermined  grading  scale  based  upon  past  HyperCard  assessment  criteria. 

Students  were  assessed  on  the  following  criteria:  (a)  HyperCard  stack  created;  (b)  backgrounds  included; 
(c)  buttons  included;  (d)  text  fields  included;  (c)  graphics  included;  (0  hyperlinks  included;  (g)  “pop-up”  items  using 
the  show/hide  commands  included;  and  (h)  the  overall  content  and  instructional  value  of  their  stack.  These  stacks 
were  graded  by  this  researcher  based  upon  preset  point  values  for  the  stated  criteria.  Stacks  were  submitted  with 
research  participant  number  as  the  only  student  identification. 

Satisfaction  Questionnaire 

A satisfaction  questionnaire  was  developed  for  administration  at  the  end  of  the  treatment.  Questions  were 
designed  to  determine  if  students  were  satisfied  with  the  learning  environment,  were  comfortable  with  the  learning 
environment,  and  to  self-assess  learning  from  the  experience.  For  every  question,  an  equal  and  opposite  question 
was  written  to  ascertain  which  wording  best  asked  the  question  to  be  answered  (e.  g.,  “I  liked  learning  HyperCard  via 
computer”  vs.  “I  did  not  like  learning  HyperCard  via  computer”).  Two  statistical  consultants  evaluated  the  questions 
for  face  and  construct  validity.  These  questions  were  then  pilot  tested  with  a group  of  approximately  sixty  subjects 
and  subsequently  scored  to  determine  how  effective  the  instrument  was  at  measuring  satisfaction.  Finally,  the 
completed  satisfaction  instrument  was  subjected  to  a Cronbach’s  Coefficient  Alpha  and  a factor  analysis  to  establish 
statistical  validity  and  reliability.  Two  of  the  initial  twenty-one  questions  were  dropped  after  the  factor  analysis 
revealed  that  these  questions  did  not  load  highly  on  any  factor  and  the  Chronbach  Alpha  procedure  revealed  that 
dropping  these  questions  increased  the  reliability  coefficient.  The  resulting  Coefficient  Alpha  for  the  satisfaction 
questionnaire  was  .89. 

Data  Collection 

Although  two  hundred  eleven  subjects  originally  took  the  GEFT,  there  was  subject  attrition  and  incomplete 
data  from  some  participants.  Therefore,  the  following  reflects  numbers  of  subjects  whose  information  was  used  in 
the  data  analysis.  One  hundred  seventy-seven  subjects  were  divided  into  field  dependent  or  field  independent 
groups  based  on  two  factors:  their  GEFT  scores  and  previous  research  (Thompson  & Knox,  1987)  which  divides 
subjects  based  on  the  GEFT  means  for  men  and  women.  Since  previous  research  (Witkin  et  a!.,  1971)  indicated 
there  were  different  means  for  males  and  females,  collecting  this  data  gave  a more  exact  description  of  the  sample 
and  provided  more  accurate  information  for  participant  assignment  to  field  dependent/independent  groups.  Ninety- 
four  field  independent  and  eighty-three  field  dependent  subjects  were  involved  in  the  treatment. 

Once  the  subjects  were  divided  into  field  dependent/independent  groups,  these  groups  were  divided  in  half 
using  a random  number  generator.  Prior  to  treatment,  one  half  of  the  field  independent  subjects  (fifty-one)  were 
made  aware  of  their  GEFT  scores  and  the  instructional  implications  of  being  either  field  dependent  or  independent. 
The  other  half  of  the  field  independent  subjects  (forty-three)  had  no  knowledge  of  their  GEFT  scores  and  did  not 
receive  information  about  the  instructional  implications  until  after  the  treatment.  Also,  one  half  of  the  field 
dependent  subjects  (forty-one)  were  made  aware  of  their  GEFT  scores  and  the  instructional  implications  of  being 
cither  field  dependent  or  independent.  The  other  half  of  the  subjects  field  dependent  subjects  (forty-two)  had  no 
knowledge  of  their  GEFT  scores  and  also  did  not  receive  information  about  the  instructional  implications  until  after 
the  treatment 

In  addition,  one  half  of  the  field  dependcnt/aware  subjects  (nineteen)  were  placed  in  a treatment  with  an 
instructional  environment  that  was  matched  with  their  cognitive  style  while  the  other  half  (twenty-two)  were 
mismatched  with  their  cognitive  style.  One  half  of  the  field  dependent/unaware  subjects  (twenty-two)  were  placed 
in  a treatment  with  an  instructional  environment  that  was  matched  with  their  cognitive  style  while  the  other  half 
(twenty)  were  mismatched  with  their  cognitive  style. 

All  volunteers  signed  up  for  a two  hour  time  slot  during  which  they  received  the  treatment.  All  subjects 
were  given  a disk  containing  the  HyperCard  treatment  specifically  designed  for  that  subject’s  assigned  group.  The 
subjects  worked  on  the  treatment  independently  and  according  to  the  specific  requirements  of  that  treatment.  They 
had  one  hour  to  complete  the  treatment  and  one  hour  to  complete  the  assessment.  The  assessment  portion  of  the 
treatment  consisted  of  the  students  developing  a HyperCard  stack  on  their  own  with  only  the  use  of  notes  taken 
during  the  treatment.  At  the  conclusion  of  the  treatment,  the  subjects  were  given  a satisfaction  questionnaire  to 
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dctennine  their  satisfaction  with  the  learning  environment,  their  comfort  with  the  learning  environment,  and  whether 
they  learned  from  the  experience. 

Analysis  Procedures 

Due  to  the  multiplicity  of  independent  variables  (field  dependcncc/independencc,  match/mismatch, 
awareness/unawareness)  and  dependent  variables  (achievement  on  the  HyperCard  stack  and  responses  to  the 
satisfaction  questionnaire),  a 2x2x2  factorial  Multivariate  Analysis  of  Variance  (MANOVA)  was  used  with  the 
alpha  level  set  at  .05. 

If  a difference  had  been  found  in  the  overall  model,  individual  univariate  procedures  would  have  been 
performed  to  identify  differences  in  main  effects  as  well  as  in  the  interactions  between  variables  for  both  dependent 
variables.  According  to  Stevens  (1992),  using  univariate  procedures  provides  the  best  power  providing  an  alpha 
level  of  .05  is  used. 

Analysis  of  Research  Questions 

This  section  is  divided  into  a discussion  of  the  data  collected  and  the  statistical  procedures  used  in  the 
analysis  of  each  of  the  research  questions.  The  research  questions  are  restated,  followed  by  an  explanation  of  the 
data,  a summary  of  the  analysis  and  findings  and  a statement  indicating  the  retention  of  the  null  hypotheses.  Tables 
arc  presented  to  display  the  statistical  data  and  levels  of  significance  when  appropriate.  Since  the  dependent 
variables  of  achievement  scores  and  satisfaction  questionnaire  scores  were  positively  correlated  (r=.5378),  a 
Multivariate  Analysis  of  Variance  (MANOVA)  was  used  for  the  statistical  analyses.  Achievement  was  measured 
using  an  achievement  rubric.  The  results  were  obtained  by  scoring  subject's  individually  created  HyperCard  stacks. 
Satisfaction  was  measured  by  obtaining  a total  satisfaction  score  based  upon  the  answers  to  the  satisfaction 
questionnaire. 

Analysis  of  the  Independent  Variables  of  Matching  and  Mismatching 

RQ,.  To  what  extent  does  matching  a student  with  a hypermedia  environment  designed  wi/i 
instructional  support  for  field  dependent  or  field  independent  cognitive  style  affect  student 
achievement  in  learning  HyperCard? 

RQ:.  To  what  extent  docs  mismatching  a student  with  a hypermedia  environment  designed  with 

instructional  support  for  field  dependent  or  field  independent  cognitive  style  affect  student 

achievement  in  learning  HyperCard? 

RQ3.  To  what  extent  does  matching  a student  with  a hypermedia  environment  designed  with 
instructional  support  for  field  dependent  or  field  independent  cognitive  style  affect  student 
satisfaction  with  the  HyperCard  lesson  and  the  learning  experience? 

RQ.}.  To  what  extent  docs  mismatching  a student  with  a hypermedia  environment  designed  with 

instructional  support  for  field  dependent  or  field  independent  cognitive  style  affect  student 

satisfaction  with  the  HyperCard  lesson  and  the  learning  experience? 

The  results  were  obtained  by  calculating  individual  subjects'  achievement  and  satisfaction  scores  and 
comparing  these  results  in  matched  versus  mismatched  group.  Results  of  the  analysis  arc  indicated  in  Table  1 . 

Table  l 

Results  of  the  Multivariate  Analysis  of  Variance  using  Wilks'  Lambda  for  the  Variables  of  Matching  and 
Mismatching  on  Achievement  and  Satisfaction  Scores 


Source 

Multivariate 

Degrees 

P 

of  Variation 

F-Ratio 

of  Freedom 

Match/Mismatch(ma/ms) 

.0669 

2;  164 

.9354 

alpha=.05 

significance  indicated  by  * 
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The  resulting  MANOVA  statistic  of  .0669  was  not  found  to  be  significant  for  the  overall  model  including 
the  dependent  variables  of  achievement  and  satisfaction.  Thus  the  null  hypotheses  were  retained.  For  this  sample, 
subjects  who  were  matched  with  an  instructional  environment  with  support  for  their  cognitive  style  did  not  score 
significantly  higher  on  either  achievement  or  satisfaction  than  those  who  were  mismatched. 

Analysis  of  the  Role  of  Awareness 

RQS.  To  what  extent  does  awareness  of  field  dcpendence/independencc  affect  student  achievement  in  a 
hypermedia  HyperCard  lesson? 

RQ6.  To  what  extent  does  awareness  of  field  dependence/independencc  affect  student  satisfaction  with  a 
hypermedia  HyperCard  lesson? 

The  results  were  obtained  by  calculating  individual  subjects'  achievement  and  satisfaction  scores  and 
comparing  these  results  for  aware  versus  unaware  groups.  Results  of  the  analysis  are  indicated  in  Table  2. 

Table  2 

Results  of  the  Multivariate  Analysis  of  Variance  using  Wilks'  Lamdafor  the  Variables  of  Aware  and  Unaware  on 
Achievement  and  Satisfaction  Scores 


Source 

Multivariate 

Degrees 

P 

of  Variation 

F-Ratio 

of  Freedom 

Aware/Unaware(aw/uw) 

.097 1 

2;  164 

.9075 

alpha=  05 

significance  indicated  by  * 


The  resulting  MANOVA  statistic  of  .0971  was  not  found  to  be  significant  for  the  overall  model  including 
the  dependent  variables  of  achievement  and  satisfaction.  Thus  the  null  hypotheses  were  retained.  For  this  sample, 
subjects  who  were  aware  of  their  cognitive  style  and  its  instructional  implications  during  the  treatment  did  not  score 
significantly  higher  on  either  achievement  or  satisfaction  than  those  who  were  unaware. 

Analysis  of  the  Interactions 

RQ7.  To  what  extent  do  these  three  variables  (learner  awarencs  of  field  dcpcndcncc/independence, 
matching/mismatching  HyperCard  lessons,  field  dependence/independencc)  singly  or  in 
combination  affect  achievement  in  learning  HyperCard? 

RQ8.  To  what  extent  do  these  three  variables  (learner  awareness  of  field  dependence/independencc. 
matching/mismatching  HyperCard  lessons,  field  dependencc/indcpendence)  singly  or  in 
combination  affect  satisfaction  with  the  HyperCard  lesson  and  the  learning  experience? 

The  results  were  obtained  by  calculating  individual  subjects'  achievement  and  satisfaction  scores  and 
comparing  these  results  in  the  interactions  between  field  dependent/independent,  matched/mismatched,  and 
awarc/unawarc  groups.  Results  of  the  analysis  are  displayed  in  Table  3. 
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Table  3 

Results  of  the  Multivariate  Analysis  of  Variance  using  Wilks’  Lamdafor  the  Interactions  on  Achievement  and 
Satisfaction  Scores 


Source 
of  Variation 

Multivariate 

F-Ratio 

Degrees 
of  Freedom 

P 

FI/FD  x AW/UW 

.0688 

2;  164 

.9336 

FI/FD  x MA/MS  • 

.0801 

2;  1 64 

.9230 

AW/UW  x MA/MS 

.4446 

2;  164 

.6418 

FI/FD  x AW/UW  x MA/MS 

.4824 

2;  164 

.6182 

alpha=.05 

significance  indicated  by  * 


The  resulting  MANOVA  statistics  were  not  found  to  be  significant  for  the  overall  model  including  the 
dependent  variables  of  achievement  and  satisfaction.  Thus,  the  null  hypotheses  were  retained.  For  this  sample, 
interaction  effects  between  the  independent  variables  of  field  dependence/indepcndence,  matching/mismatching  and 
awarc/unaware  were  not  found  to  make  a significant  difference  in  achievement  or  satisfaction  scores. 

Other  Data  Analyses 

Although  the  statistical  procedures  defined  by  the  research  design  and  methodology  were  used,  statistical 
significance  for  the  hypotheses  was  not  obtained.  However,  it  may  be  important  to  document  other  types  of 
descriptive  and  qualitative  data  for  further  research. 

Means  and  Standard  Deviations 

There  are  times  when  using  advanced  statistical  procedures  yields  little  in  the  explanation  of  what  occured 
during  data  collection.  Therefore,  an  examination  of  means  and  standard  deviations  may  document  the  potential  for 
any  unsubstantiated  group  differences.  These  descriptive  statistics  are  listed  in  Tables  4 and  5. 


Table  A 

Means  and  Standard  Deviations  for  the  Dependent  Variable  of  Achievement 


Group 

M 

Achievement 

SD 

n 

FD/AW/MA 

5.1842 

2.1616 

19 

FD/AW/MS 

5.0000 

2.3350 

22 

FD/UW/MA 

5.3409 

1 .9844 

22 

FD/UW/MS 

4.1000 

1 .8680 

20 

FI/AW/MA 

4.6875 

1 .9826 

24 

FI/AW/MS 

4.5909 

2.7802 

22 

FI/UW/MA 

4.9815 

2.1505 

27 

FI/UW/MS 

5.9048 

2.3644 

21 
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Table  5 

Means  and  Standard  Deviations  for  the  Dependent  Variable  of  Satisfaction 


Group 

M 

Satisfaction 

SD 

n 

FD/AW/MA 

46.6111 

17.3651 

19 

FD/AW/MS 

46.8571 

16.9035 

22 

FD/UW/MA 

40.9090 

15.3837 

22 

FD/UW/MS 

41.6500 

14.4086 

20 

FI/ AW/M  A 

44.5000 

14.6495 

24 

FI/AW/MS 

44.0682 

18.1267 

22 

FI/UW/MA 

45.5185 

14.0757 

27 

FI/UW/MS 

53.4048 

17.2740 

21 

The  overall  means  in  the  case  of  this  treatment  were  low,  both  for  achievement  and  satisfaction.  However, 
all  eight  groups  had  individuals  with  higher  scores,  middle  range  scores,  and  lower  scores.  Note  that  the  means  and 
standard  deviations  are  similar  for  all  groups.  There  may  be  other  variables  affecting  achievement  and  satisfaction 
different  from  the  independent  variables  analyzed  in  this  study. 

Analysis  of  Qualitative  Data 

Although  the  quantitative  data  indicated  non-significant  results,  the  qualitative  data  obtained  via  the  open- 
ended  question  on  the  satisfaction  questionnaire  and  from  post-treatment  conversations  with  subjects  yielded 
important  information.  A number  of  field  dependent  subjects  indicated  that  they  believed  it  was  important  for  them 
to  have  extensive  instructional  support  with  step-by-step  instruction  (a  few  subjects  directly  quoted  the  field 
dependent/independent  information  given  to  them  during  the  instruction).  Several  of  the  subjects  who  were 
mismatched  with  their  instructional  environments  were  very  upset  by  this  mismatching  and  confessed  they  were 
confused  and  frustrated.  Those  who  were  aware  of  the  instructional  implications  of  being  fieid  dependent 
questioned  why  they  were  not  given  a treatment  with  more  support  and  directions.  Those  who  were  unaware  of 
their  cognitive  style  blamed  themselves  for  their  failurc--many  stating  that  were  “no  good  at  timed  tests”  or  were 
“stupid.”  The  most  outspoken  subjects  seemed  to  be  in  the  field  dependent,  unaware,  mismatched  group. 

On  the  other  hand,  the  field  independent,  aware,  mismatched  group  was  also  more  outspoken  than  subjects 
in  other  groups.  The  comments  for  the  field  independent  subjects  differed  greatly  from  those  of  the  field  dependent 
subjects.  Comments  made  by  this  group  included  such  things  as  “the  instruction  was  loo  casy-I  thought  HyperCard 
would  be  easy,”  and  “it  drove  me  crazy  to  go  slcp-by-slep  like  that-I  just  got  through  the  instruction  as  soon  as 
possible.”  These  subjects  had  a tendency  to  blame  external  sources  for  their  perceived  lack  of  success.  Note  taking 
was  commonly  cited  as  a problem  for  this  group.  Subsequent  review  of  notes  indicated  that  the  subjects  actually 
look  little  if  any  notes  and  did  not  write  down  information  specifically  mentioned  as  important  for  their  ability  to 
create  their  own  stacks  during  the  second  part  of  the  treatment. 

There  were  some  other  interesting  overall  trends  pertaining  to  field  dependent  versus  independent  subjects 
regardless  of  awareness  or  matching.  Many  field  dependent  subjects  suggested  their  perceived  failure  on  the  task 
was  due  to  human-centered  difficulties  with  the  treatment.  Several  suggested  that  they  needed  more  support  from 
the  instructor , not  the  computer.  One  subject  made  the  statement  that  it  was  important  to  learn  via  a social  learning 
environment  (quoting  information  given  during  the  treatment);  from  human  bcings--not  a computer. 

Field  independent  subjects  seemed  to  desire  more  access  to  external  resources.  These  subjects  seemed  to 
like  learning  via  computer  but  vould  have  preferred  to  be  able  to  ask  questions  of  the  researcher  during  the  second 
portion  of  the  treatment.  The  qualitative  information  seems  to  be  in  agreement  with  researchers  such  as 
Goodenough  (1976). 

Discussion  and  Suggestions  for  Future  Research 

The  results  of  this  study,  in  conjunction  with  the  results  from  other  studies,  do  not  support  conducting 
additional  quantitative  research  to  measure  complex,  interrelated  dependent  learner  variables.  Separating  subjects 
into  groups  in  order  to  isolate  variables  did  not  yield  as  much  useful  information  as  did  the  free  response  data 
collected  with  the  satisfaction  questionnaire.  Future  researchers  should  concentrate  on  qualitative  methodologies  to 
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examine  how  individuals  interact  with  learning  environments.  In  the  next  section,  suggestions  for  future  research 
will  be  discussed. 

Qualitative  Research 

This  researcher  believes  that  it  is  important  to  conduct  qualitative  research  concerning  the  ways  in  which 
field  dependent  and  field  independent  individuals  interact  with  learning  environments.  As  previously  mentioned, 
although  it  was  specifically  stated  in  the  initial  instructions  that  they  would  be  receiving  their  results  during  the 
treatment,  why  were  the  vast  majority  of  subjects  who  claimed  they  did  not  receive  their  results  field  independent? 
Js  there  something  instructors  can  do  to  “force”  field  independent  subjects  to  read  important  information  during 
independent  learning  tasks?  Did  the  subjects  not  listen  to  the  instructor  or  read  the  information  because  they 
perceived  it  to  be  irrelevant  to  the  learning  task  at  hand?  These  are  questions  which  may  be  answered  by  future 
research. 

For  field  dependent  subjects,  is  it  possible  for  them  to  learn  and  to  feel  comfortable  in  an  independent, 
computer  based  learning  environment?  Many  field  dependent  individuals  commented  on  the  satisfaction 
questionnaire  that  they  would  have  been  more  successful  had  they  learned  from  a “real”  person  (not  a computerized 
treatment).  Although  the  treatment  was  structured  to  include  the  adaptations  for  cognitive  style  during  the 
instructional  portion,  subjects  worked  through  the  HyperCard  treatment  on  an  individual  computer.  There  was  no 
group/social  interaction. 

Future  researchers  might  want  to  consider  designing  an  environment  with  extensive  resources  and  observe 
how  field  dependent/independent  subjects  select  those  resources.  Also,  they  should  investigate  which  resources  arc 
important  to  learners  and  if  resource  selection  varies  by  degree  of  field  dependence/independence.  However,  to  be 
effective  computer  based  instruction  (CBJ),  extensive  front-end  analysis,  formative  evaluation,  as  well  as  special 
design  considerations  are  required.  This  is  not  always  practical  for  a single  individual  to  attempt  in  their  own 
classroom.  Instead,  including  resources  such  as  a live  instructor,  print  materials  or  other  media  from  an  outside 
source,  or  peer  tutors,  in  addition  to  the  CBI  may  be  a practical  and  cost  effective  way  of  designing  learning 
environments  which  support  both  field  dependence/independence. 

Cognitive  Restructuring 

Witkin  & Goodenough  (1981)  discuss  “cognitive  restructuring”—  the  ability  of  some  individuals  to 
restructure  the  field  depending  upon  the  task.  After  this  current  treatment,  several  individuals  confided  in  this 
researcher  that  they  did  not  believe  the  information  regarding  their  Group  Embedded  Figures  Test  score  fit  them  at 
all.  Upon  further  examination,  many  of  these  individuals  had  GEFT  test  scores  which  placed  them  along  the  mean. 
Thus,  these  individuals  were  neither  strongly  field  dependent  nor  strongly  field  independent.  The  individuals  were 
then  asked  whether  or  not  they  believed  the  information  fit  them  when  interacting  with  any  learning  environment. 
The  majority  of  individuals  said  that  in  some  cases  the  information  reflected  tt  ways  in  which  they  usually  learned 
but  definitely  not  all  the  time.  In  the  future,  it  might  be  important  to  determine  to  what  degree  individuals  whose 
GEFT  scores  fall  along  the  mean  are  able  to  change  learning  strategies  and  under  which  conditions  they  make  this 
change 

Job  Related  Cognitive  Style  Information 

Although  many  studies  have  shown  field  independent  individuals  to  outperform  field  dependent  individuals 
regardless  of  the  task  (e.g.,  Bishop-Clark  ,1992;  Frank,  1984;  Jonassen,  1980),  there  may  be  other  factors  which 
impact  performance,  especially  on  the  job.  Factors  such  as  missed  deadlines,  absenteeism  and  the  ability  to  follow 
directions  might  have  as  great  an  impact  on  job  performance  as  the  overall  quality  of  one’s  work.  This  point  is 
mentioned  because  at  one  point  during  the  ten  days  over  which  the  treatment  was  conducted,  field  independent 
subjects  were  four  times  as  likely  not  to  take  the  treatment  during  their  originally  scheduled  times  as  field  dependent 
subjects  (actual  numbers:  3 to  12).  It  might  be  of  interest  to  future  researchers  to  examine  the  role  of  field 
dcpcndc.ncc/indcpendence  on  overall  job  performance. 

The  Perfectionist  Factor 

Although  achievement  and  satisfaction  were  shown  to  he  moderately  correlated  overall,  there  was  a select 
group  of  subjects  (twenty-eight)  for  which  this  was  not  the  case.  For  these  individuals,  a fairly  high  achievement 
(7.5  or  greater)  was  paired  with  a very  low  satisfaction  score  (50  and  below).  Upon  further  examination  of  the 
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satisfaction  questionnaire  data  about  the  reasons  they  did  not  believe  they  had  satisfactorily  completed  the  task, 
many  of  these  subjects  indicated  that  they  were  unable  to  accomplish  some  small  portion  of  the  overall  instructional 
goal  (e.g.,  “I  could  not  do  pop-ups”  or  “I  could  not  get  my  quit  button  to  work’*).  It  did  not  appear  to  this  researcher 
that  there  was  a pattern  to  the  characteristics  of  these  subjects.  It  might  be  important  to  consider  why  these 
individuals  were  so  dissatisfied  with  a better  than  average  performance. 

The  Design  of  Hypermedia 

Empirical  results  indicate  that  making  subjects  aware  of  their  cognitive  style  and  matching  or  mismatching 
them  with  instruction  embedded  with  support  for  cognitive  style  did  not  make  a difference  for  this  sample. 
Comments  from  students  may  be  important  in  the  design  of  future  hypermedia  environments.  This  researcher  has 
the  following  suggestions  related  to  the  use  of  hypermedia  learning  environments: 

1.  Many  subjects  may  need  additional  support  regardless  of  cognitive  style  when  completing  complex 

tasks. 

2.  Support  does  not  always  correlate  with  structure.  Many  fi  'Id  independent  subjects  did  not  like  the 
structure  of  the  field  dependent  treatment.  They  would  have  liked  to  ask  questions  of  the  researcher. 

3.  Providing  plenty  of  resources,  according  to  Jonassen  and  Grabowski  (1993),  may  be  important  for  field 
independent  subjects.  However,  it  may  also  be  important  to  provide  similar  resources  for  field  dependent  subjects. 
For  example,  including  suggestions  for  instructional  paths  in  a hypermedia  environment  which  can  be  printed  would 
provide  another  “resource”  for  field  independent  subjects  and  provide  “structure”  for  field  dependent  subjects. 
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Abstract 

As  the  number  and  complexity  of  required  technology-related  competencies  grow,  basic  teacher  education 
media  courses  need  to  develop  alternatives  to  instructor-led  lecture-based  formats  in  order  to  maintain  essential 
components  of  course  content  while  incorporating  new  content  in  expanding  technologies.  Creating  technology- 
mediated  instruction  that  are  used  as  out-of-class  assignments  is  one  way  to  deal  with  this  issue . This  paper  presents 
the  process  of  transforming  lesson  content  from  lecture-based  format  to  instructional  content  delivered  through  the 
World  Wide  Web  It  details  the  steps  taken  to  make  the  Web  site  instructional  and  interactive  rather  than  merely 
informational. 

Introduction 

The  dilemma  wc  all  face  in  delivering  technology-based  courses  is  how  to  add  new  technology  information 
and  skills  while  maintaining  essential  existing  content.  Alternative  instructional  delivery  mechanisms  must  be 
explored  to  deal  with  the  dilemma. 

The  World  Wide  Web  is  host  to  hundreds  of  thousands  of  Web  sites.  Information  is  accessible  day  and 
night,  anywhere  in  the  world,  on  any  topic.  The  Web  is  also  becoming  increasingly  used  for  instructional  purposes, 
not  just  information  delivery.  The  accessibility  of  information  and  the  ability  to  create  interactive  instruction  on  the 
Web  make  it  an  attractive  medium  for  the  delivery  of  instruction. 

Framework 

Courses  in  educational  media  are  often  logical  places  to  incorporate  new  and  expanding  technology  content 
for  undergraduate  education  majors.  At  our  university,  technology  components  primarily  reside  in  one  educational 
media  course.  As  we  have  expanded  the  course  content  over  time  beyond  planning,  creating,  evaluating,  and  using 
instructional  media  to  include  word  processing,  graphics,  multimedia,  and  the  Internet,  existing  content  is 
continually  revised  and  updated.  By  critically  analyzing  course  content,  components  arc  identified  as  possible 
candidates  for  alternative  instructional  delivery.  In  a continuing  effort  to  develop  mediated  instruction,  selected 
components  of  the  educational  media  course  arc  targeted  for  implementation  in  alternative,  computer-mediated 
formats. 

Visual  literacy  was  identified  as  one  course  component  that  might  alternatively  be  taught  through  a 
technology-based  medium  rather  than  through  instructor-led  classroom  lecture.  Since  teachers  need  to  be  able  to 
understand,  interpret,  and  create  visual  images  for  instructional  purposes,  a basic  understanding  of  visual  literacy 
concepts  is  crucial  not  only  in  planning  and  developing  visual  images,  but  also  in  evaluating  images  for  instruction. 
A technology-based  instructional  package  on  visual  literacy  would  give  students  the  capability  to  learn  visual 
literacy  concepts  as  an  out-of-class  assignment  that  could  then  be  reinforced  through  classroom  discussion  and 
course  assignments,  thus  freeing  up  the  contact  time  for  other  issues. 

Defining  Visual  Literacy  Content 

Visual  literacy  is  a required  topic  in  the  undergraduate  coursework  for  students  preparing  for  the  teaching 
profession  at  our  universitv.  We  are  bombarded  with  visual  messages  from  television,  film,  books,  periodicals, 
billboards,  and  computer  Sv.  ‘cns.  Painters,  sculptors,  actors,  dancers,  and  other  artists  also  visually  communicate 
messages  to  us.  Educators  need  to  be  able  to  understand  how  to  communicate  visually  with  their  students  as  well  as 
how  to  help  their  students  interpret  what  they  sec.  While  entire  course'1  can  be  devoted  to  the  study  of  visual  literacy, 
in  preservice  education  the  need  is  to  focus  on  basic  principles  and  key  factors  rather  than  the  entire  scope  of  the 
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discipline.  The  conleni  of  the  lesson  in  this  course  includes  an  introduction  to  basic  visual  literacy  concepts,  visual 
design  elements  and  principles,  factors  that  impact  visual  message  interpretation,  and  practical  guidelines  for 
producing  instructional  visual  images.  This  content  reflects  topics  on  visual  literacy  principles  from  three  sources: 
Instructional  Media  and  Technologies  for  Learning  (Hcinich,  Molcnda,  Russell,  & Smaldino,  1996),  Visual 
Literacy:  A Spectrum  of  Visual  Learning  (Moore  & Dwyer,  1994),  and  Visual  Communicating  (Wileman,  1993). 

Design  of  Computer-mediated  Instruction 

An  instructional  design  was  completed  (Kovalik  & Sboffner,  1996)  to  make  the  transition  from  an 
instructor-led  lecture-based  format  to  a computer-mediated  format.  A decision  was  made  to  create  the  instruction  in 
two  formats,  as  a computer-based  tutorial  and  as  a World  Wide  Web  site. 

After  completing  the  instructional  design,  storyboards  were  created  for  computer-based  instruction  (CBI) 
on  visual  literacy.  A subject  matter  expert  and  an  instructional  technology  graduate  student  reviewed  the 
storyboards.  Suggestions  for  changes  were  discussed  with  the  designer  and  revisions  made  where  appropriate  and 
necessary.  Each  CBI  screen  was  designed  to  contain  a minimum  amount  of  text  with  a related  visual  image.  The 
images  wore  selected  to  clarify  or  illustrate  the  accompanying  text  (sec  Figure  1).  All  images  used  in  the  instruction 
were  either  copyright-  free  clip  art  or  original  work  of  the  authors.  Obtaining  permission  to  use  copyrighted  work 
was  loo  time  consuming  and  potentially  too  costly.  While  there  are  images  that  would  better  illustrate  a concept,  if 
they  were  not  available  copyright-free  they  were  not  considered  for  inclusion  in  the  instruction. 

Figure  L Visual  Literacy  Website  Screen  Dealing  With  the  Design  Element , Color 


Design  Elements  - Color 

Color  is  the  part  of  light  that  is  reflected  \>y  the 
object  ve  see 


Following  the  design,  development,  and  implementation  of  the  computer-based  tutorial,  initial  formative 
evaluation  was  conducted  with  students  in  a graduate  instructional  technology  course.  Changes  were  made  to  correct 
minor  deficiencies,  but  no  major  content  changes  were  made.  The  computer-based  tutorial  was  translated  to  HTML 
and  uploaded  to  a Web  site.  The  Web  version  mirrored  the  CBI  version. 
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Research  Results 

The  effectiveness  of  newer  interactive  technology  usage  in  instruction  is  open  to  debate.  Moore,  Myers, 
and  Burton  (1993)  state  that  little  actual  research  regarding  the  effectiveness  of  interactive  technologies  has  been 
conducted  and  what  has  been  conducted  has  limited  value.  Practitioners  and  researchers  call  for  research  that  goes 
beyond  traditional  comparison  studies.  Grabowski  and  Pearson  (1988),  Sice  (1989),  Reeves  (1986),  Kozina  (1991), 
and  Moore  et  al  (1993)  have  made  calls  for  conducting  research  on  the  instructional  strategies  related  to  the  specific 
attributes  of  the  interactive  medium.  Hannafin  (1992)  identified  the  areas  of  constructivist  models  of  teaching  and 
acquiring  knowledge,  hypermedia,  and  cooperative  teaching  and  learning  as  innovative  instructional  strategies  that 
need  to  be  addressed  in  emerging  technology  research. 

In  keeping  with  these  recommendations  for  examining  instructional  strategies  in  technology-mediated 
instruction,  a research  study  was  designed  to  determine  the  effects  of  varying  instructional  strategies  on  visual 
literacy  instruction  (Shoffncr,  1997;  Shoffner  & Dalton,  1998).  The  study  used  a completely  crossed  2 
(organizational  strategy)  x 2 (delivery  strategy)  x 2 (management  strategy)  factorial  design,  post-test  only  control 
group  design.  Organizational  strategies  employed  included  criterion-referenced,  objectivist  instruction  and  problem- 
based,  constructivist  instruction.  Delivery  strategies  employed  included  the  local,  computer-based  instruction  and 
the  web-based  instruction  described  above.  Two  management  strategies  were  used,  participation  by  individuals  and 
cooperative  dyads.  One  hundred  thirty-eight  undergraduate  education  students  were  randomly  assigned  to  treatment 
groups  and  a control  group  and  received  self-paced  instructional  treatments.  Two  weeks  after  treatment,  section 
instructors  collected  subject-created  instructional  media  (poster  and  overhead  transparency).  At  that  time,  each 
subject  completed  a 30-item  objective  achievement  instrument  during  a regular  class  session.  Subject  instructional 
media  products  were  assessed  using  a checklist  that  rated  appropriate  use  of  visually  literacy  concepts.  Results  from 
these  two  measures  were  combined  to  produce  a measure  of  achievement.  Learners  were  assessed  on  performance 
(achievement),  and  instructional  efficiency  (achievement  divided  by  time  spent  on  treatment). 

achievement  measures  for  all  treatment  groups  showed  significantly  higher  levels  of  achievement  when 
compared  to  the  appended  control  group.  Using  multivariate  and  univariate  analyses  of  variance,  data  were 
examined  to  determine  what  effect  varying  instructional  strategies  had  on  learning  (achievement  and  instructional 
efficiency).  Results  indicated  no  statistically  significant  differences  between  any  treatment  groups.  In  particular, 
results  indicated  no  significant  differences  in  achievement  between  the  computer-based  tutorial  and  the  web-based 
instruction. 

This  result  was  not  unanticipated  since,  in  an  attempt  to  control  for  error,  design  of  the  Web  versions  was 
highly  similar  to  the  respective  CBI  versions.  While  initial  results  indicated  that  the  CBI  and  web-based  instruction 
were  equal  in  achievement  results,  wc  decided  to  maintain  only  the  Web  site.  There  arc  several  reasons  for  this 
decision:  (a)  Students  have  consistently  commented  in  course  evaluations  that  the  class  requires  too  much  time  for 
the  credit  hours.  This  perception  is  largely  due  to  the  fact  that  because  of  specialized  equipment  and  materials, 
students  must  spend  much  of  their  out-of-class  time  in  labs  to  complete  assignments  and  open  lab  times  are  not 
always  convenient  to  student  schedules,  (b)  One  of  the  principles  followed  in  the  course  is  to  demonstrate  the 
technology  through  a variety  of  instructional  delivery  methods  as  well  as  instructing  about  media  and  technology, 
(c)  Some  Instructional  Technology  graduate  courses  are  being  redesigned  for  distributed  education.  Maintaining  this 
module  on  the  Web  makes  it  accessible  to  students  in  these  courses  as  well  as  the  Principles  of  Educational  Media 
students,  (d)  Many  of  the  assignments  in  the  courses  arc  dependent  on  the  visual  literacy  concepts  presented  on  the 
Web.  The  information  needs  to  be  available  on  an  ongoing  basis. 

Using  web-based  instruction  allows  students  to  access  the  site  at  their  convenience,  demonstrates  an 
innovative  instructional  technique,  makes  it  accessible  to  students  in  distance  learning  classes,  and  makes  it 
available  as  a reference  while  completing  assignments.  The  CBI  version  cannot  address  all  of  these  issues  as  easily 
as  the  web-based  instruction. 

Student  Feedback 

The  original  Web  site  provided  basic  visual  literacy  information.  Even  though  there  arc  several  places  in 
the  site  where  users  arc  queried  on  the  interpretation  of  visual  images  (see  Figure  2),  there  is  no  mechanism  to 
accept  user  input  and  provide  feedback.  Concerned  about  this  lack  of  interactivity,  we  continually  sought  formal  and 
informal  evaluative  comments  and  criticism  of  the  site  from  users.  After  students  completed  the  Web  assignment, 
they  were  asked  to  react  to  the  site  in  terms  of  its  instructional  effectiveness  and  usability. 

Figure  2.  Types  of  Rhetorical  Questions  Asked  in  the  Original  Website 
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Try  this  exercise.  Look  at  each 
■umbrella  image.  Is  the  message  of 
each  image  different?  If  so,  what 
makes  the  message  different? 


* 4 

T 


/ 


Here  ere  three  more  umbrella  images.  How  do 
these  images  compare  to  each  other?  Ho w do 
these  images  compare  with  the  three  previous 
ones? 


What  happens  whe n the  image  of  the  umbrella  includes  a 
person?  What  changes  about  the  visual  message?  Axe  there 
differences  between  these  two  images?  If  so,  what  are  they? 


Our  results  indicate  that,  overall,  users  like  the  Web  site.  Most  feel  the  site  is  well  organized,  self- 
explanatory,  and  contains  an  adequate  amount  of  basic  information  on  visual  literacy.  A number  of  users  felt  they 
benefited  from  tcxt/graphic  combinations.  The  ability  to  sec  the  words  and  concepts  while  having  the  graphic  to 
reinforce  the  ideas  presented  was  appreciated.  User-control  was  also  cited  as  a positive  attribute  of  the  Web  site. 
Students  felt  that  the  ability  to  work  at  their  own  pace  and  the  ability  to  review  the  information  were  important  to 
their  learning. 

The  ability  to  access  the  tutorial  at  their  convenience  and  to  print  the  information  were  also  cited  as 
advantages  of  the  web-based  instruction.  As  indicated  previously,  student  complaints  about  the  work  involved  in  the 
course  centered  around  the  need  for  lab  time.  The  availability  of  the  instruction  at  any  time  is  an  advantage  for 
learners.  Students  also  commented  on  the  change  from  a classroom  Iccturc-bascd  format  to  the  self-instructional 
formal.  When  information  is  presented  in  lecture  format,  some  students  concentrate  on  writing  comprehensive  notes 
rather  than  on  listening  to,  and  trying  to  understand,  content.  With  the  web-based  instruction,  students  can  review 
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the  information  as  needed.  When  students  arc  equipped  with  content  knowledge,  class  discussion  and  reinforcement 
of  the  content  becomes  more  meaningful.  Previously,  class  time  was  devoted  solely  to  presentation  of  visual  literacy 
content.  Within  that  framework,  students  applied  visual  literacy  concepts  to  projects  with  varying  success.  Since  the 
introduction  of  the  web-based  visual  literacy  site,  there  has  been  a dramatic  positive  increase  in  the  quality  of 
student-created  projects.  This  increase  may  be  partly  attributable  to  classroom  discussion,  reflection,  and  critique  of 
visual  messages. 

Students  also  provided  suggestions  for  improvement.  The  majority  of  suggestions  focused  on  the  lack  of 
interactivity.  Students  indicated  they  would  like  to  experiment  with  basic  design  elements  and  principles.  They  also 
commented  that  there  is  a need  for  interactivity  and  feedback  from  the  site.  The  perception  was  that  they  could  have 
retained  more  if  it  was  more  interactive. 

This  emphasis  on  lack  of  user  interaction  from  the  siudents  and  our  own  recognition  that  the  Web  could  be 
more  interactive  caused  us  to  look  for  ways  to  provide  interactivity  and  feedback.  Although  questions  were  asked  to 
encourage  the  student  to  think  about  further  applications  of  the  information,  we  had  not  provided  the  learner  with 
meaningful  ways  to  test  his/her  understanding  of  the  concepts  or  to  practice  visual  literacy  skills.  Interactive 
exercises  would  greatly  improve  the  site.  We  therefore  began  to  address  the  identified  problems  and  recommend 
revisions. 

Interactive  Instruction 

Changes  in  technology  have  advanced  the  potential  use  of  the  Web  beyond  sources  of  information  to 
interactive  instructional  mediums.  Links  to  related  sites,  areas  to  input  data,  and  ways  to  send  information  to  and 
from  multiple  sites  offer  one  type  of  interactivity,  but  there  are  other  possibilities. 

Our  goal  was  to  modify  the  visual  literacy  Web  site  to  make  it  more  instructional  and  interactive. 
Instructional  strategies  designed  to  improve  learning  include  embedding  problem-solving  activities,  enabling 
question-and-answer  sessions,  and  adding  evaluation  measurements.  Looking  first  to  interactivity,  the  original  visual 
literacy  Web  site  included  only  two  interactive  functions,  pacing  and  navigation.  Our  goal  was  to  add  the  functions 
of  confirmation  and  inquiry.  Additionally,  we  investigated  computer-mediated  testing  options  including  single 
multiple-choice,  multiple  multiple-choice,  single  point-and-click,  multiple  point-and-click,  free  response,  and 
simulation  (Foster  and  Olsen,  1997). 

We  First  concentrated  on  ways  to  add  interactivity  to  the  Web  site.  Several  strategies  were  identified, 
including  providing  a realistic  problem  to  be  solved  using  visual  literacy  concepts,  displaying  multiple  images  and 
asking  students  to  identify  relevant  visual  literacy  elements,  and  providing  a visual  representation  of  a concept  to  be 
interpreted  from  multiple  cultural  perspectives.  Options  for  revision  of  the  visual  literacy  Web  site  ranged  from  a 
complete  reworking  of  the  site,  to  the  addition  of  a separate  path  for  activities,  to  making  minor  modifications,  such 
as  adding  a multiple  choice  quiz.  Recognizing  the  strengths  of  the  existing  Web  site,  we  decided  to  build  two  paths. 
One  path  would  maintain  the  current  content,  the  second  path  would  provide  the  learner  with  a scries  of  problem- 
based  options.  In  this  way  we  felt  we  could  best  accommodate  differing  learning  styles. 

Allowing  learners  to  determine  their  own  learning  strategies  may  be  preferable  to  over-structuring  that 
experience  (Kinzie  & Berdel,  1990).  As  noted  by  Kovalik  & Dalton  (1997), 

One  way  learners  structure  their  learning  experiences  is  by  exercising  control  over  various  lesson  attributes 
including  instructional  sequence,  pacing,  and  options.  Learner  control  removes  the  learner  from  a 
prescribed,  regimented,  predetermined  lesson  sequence  and  allows  freedom  of  movement  in  and  between 
lesson  components.  This  freedom  often  enables  the  learner  to  structure  the  learning  experience  in  a way 
that  may  best  facilitate  fusing  new  information  with  preexisting  knowledge  (p.  162). 

Although  the  Web  site  allows  for  movement  between  lesson  components,  other  learning  options  were  not 
available.  The  addition  of  a path  containing  relevant  exercises  provides  navigational  choice.  A learner  can  elect  to 
traverse  the  Web  site  in  a linear  fashion  and  then  apply  knowledge  gained  by  working  through  the  exercises,  or, 
alternatively,  a learner  can  initially  try  the  exercises  and  refer  back  to  the  tutorial  to  search  for  answers  and  gain 
clarification. 

Designing  the  Instruction 

The  discussion  next  focused  on  designing  instruction  to  meet  the  needs  of  our  target  audience, 
undergraduate  teacher  education  majors.  Of  particular  concern  was  the  need  to  provide  these  learners  with  the 
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ability  to  evaluate  and  interpret  visuals  as  well  as  to  create  instructional^  effective  visuals,  such  as  overhead 
transparencies  and  classroom  handouts.  We  therefore  wanted  to  incorporate  meaningful  problems  that  dealt  with 
both  creation  and  interpretation. 

While  it  would  be  possible  to  find  instructional  activities  and  images  that  are  already  created  for  this  purpose, 
avoiding  copyright  complications  again  became  an  issue.  Al$o  considered  was  linking  to  existing  visual  literacy 
Web  sites  for  instruction.  There  are  a number  of  visual  literacy  sites  which  could  have  been  used  for  information  but 
none  that  we  found  with  instructional  activities.  Wc  also  rejected  linking  to  established  sites  because  of  the  possible 
navigation  difficulties  it  would  present  the  learners,  the  potential  wait  time  for  downloading,  and  the  possibility  of 
sites  not  being  available.  As  Web  utilization  capabilities  change,  this  issue  might  be  revisited. 

The  decision  was  made  to  develop  original  activities  and  possible  problem-solving  activities  were 
discussed.  Guided  by  a self-paced  instructional  module  (Kovalik,  1995),  we  brainstormed  ideas  to  identify 
appropriate  activities.  Five  activities  were  selected  for  possible  inclusion  on  the  Web  site.  These  activities  arc: 
critiquing  a handout,  identifying  design  elements  and  design  principles,  creating  a visual  message,  selecting 
appropriate  visual  messages,  and  perceptual  interpretations  of  visual  messages.  Plans  are  to  develop  all  of  these 
activities;  however,  only  two  have  been  completed  to  date.  Work  continues  on  the  remaining  exercises  which  will  be 
added  to  the  site  when  completed.  The  two  completed  activities  arc  described  here. 

Activity  1 - Critique  A Handout  Based  On  Visual  Literacy  Concepts 

This  exercise  was  based  on  an  exercise  created  by  Annette  Lamb  (1996).  Dr.  Lamb  graciously  allowed  its 
inclusion  on  the  Web  site.  In  this  activity,  the  user  is  presented  with  the  image  of  a handout  that  contains  errors  (see 
Figure  3).  The  user  is  then  asked  to  critique  the  handout  by  clicking  on  a portion  of  the  handout.  Each  selectable 
portion  of  the  image  may  contain  multiple  visual  literacy  errors. 


Figure  3.  Exercise  to  Critique  a Handout 


Check  all  thfct  ftpply  to  ihc  area  you  selected; 
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After  selecting  an  area  to  critique,  users  sec  a list  of  possible  errors.  The  user  clicks  on  all  errors  that  apply 
to  the  selection.  This  approach  incorporates  a multiple  multiple-choice  evaluation  strategy.  This  multiple  multiple- 
choice  test  strategy  enables  a broader  evaluation  of  learner  conceptual  understanding  since  the  learner  must  consider 
all  options  as  possible  correct  responses  rather  than  eliminating  options  until  only  one  “best’”  answer  remains. 
Feedback  is  based  on  choices  made  by  the  user.  If  the  user  made  inaccurate  or  inappropriate  responses,  the  feedback 
provides  an  explanation  of  why  another  choice  is  more  appropriate.  Users  choosing  correct  responses  are 
congratulated.  Following  this  activity,  the  user  is  presented  with  the  option  of  viewing  revised  handouts  and 
accompanying  explanations  of  their  respective  strengths  and  weaknesses  for  instructional  purposes,  (see  Figure  4). 
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Figure  4.  Strengths  and  Weaknesses  of  the  Handout 


Exercise  2 

Here  are  some  possible  revisions  to  the  Geography  of  North  Carolina  handout 
Click  on  the  one  that  you  think  is  most  effective. 


Geography  of  Worth  Carolina 
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Activity  2 - Identify  Design  Elements  And  Design  Principles 

Two  graphic  images  arc  presented  to  the  user  that  illustrate  a visual  literacy  concept.  The  user  is  asked  to 
indicate  which  visual  literacy  concept,  element,  or  principle  the  image  represents.  The  matching  between  image  and 
design  element/design  principle  exercise  is  designed  to  reflect  a single  multiple-choice  evaluation  strategy.  For  each 
image,  the  user  is  given  a selected  list  of  design  elements  or  design  principles.  The  user  clicks  on  the  design  clement 
or  the  design  principle  that  the  image  represents  (see  Figure  5).  Feedback  consists  of  informing  the  user  as  to  the 
correctness  of  his/her  response  and  an  explanation  of  misinterpretations  where  appropriate. 
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figure  5.  Matching  an  Image  to  a Design  Principle  ora  Design  Element 

Visual  Literacy  Question.  2 
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The  Visual  Literacy  Instructional  Package 

We  are  currently  evaluating  the  effectiveness  of  the  two  additions  to  the  site  while  developing  additional 
ones.  We  are  confident  that  the  proposed  changes  will  improve  the  instructional  effectiveness  of  the  Web  site.  The 
use  of  examples  and  nonexamplcs  has  been  an  effective  instructional  strategy  in  face-to-face  classroom  instruction. 
We  have  seen  improvement  in  student-produced  media  by  employing  this  technique  in  the  educational  media 
course.  In  the  classroom,  examples  of  student-created  posters  and  overhead  transparencies  arc  critiqued.  The  ensuing 
classroom  discussion  provides  reinforcement  of  learned  visual  literacy  concepts.  Our  goal  is  to  provide  similar 
reinforcement  through  the  interactive  exercises  on  the  Web  site.  While  it  is  important  to  provide  students  with  a 
basic  conceptual  understanding  of  visual  literacy,  that  is,  to  provide  them  with  information,  it  is  equally  important  to 
provide  ways  they  can  practice  using  this  knowledge.  It  is  not  enough  for  preservicc  teachers  to  be  able  to  simply 
recall  or  recognize  correct  answers  about  visual  literacy  on  a criterion-referenced  test.  In  preparing  to  become 
teachers,  undergraduate  education  majors  must  be  given  the  opportunity  to  actively  use  and  incorporate  visual 
literacy  concepts  in  order  to  strengthen  their  visual  literacy  skills. 

Conclusion 

Modifying  existing  preservice  teacher  education  programs  to  incorporate  new  content  reflecting  a growing 
emphasis  on  technology  requires  identification  of  instructional  strategies  that  allow  for  maintaining  existing  course 
content  while  adding  new  content.  Technology-mediated  instructional  delivery  is  one  alternative  to  classroom 
contact  time.  Transforming  course  content  to  self-contained,  self-paced,  computer-mediated  formats  gives  students 
the  opportunity  to  learn  content  through  out-of-class  assignments  at  their  convenience.  Using  the  World  Wide  Web 
as  the  delivery  mechanism  increases  accessibility  for  students.  Instruction  on  the  Web,  however,  must  be  more  than 
informatk  Students  need  to  be  given  opportunities  to  practice  their  knowledge  using  content  in  meaningful  ways. 
Additionally,  students  need  feedback  related  to  their  progress. 

The  capability  of  the  Web  to  provide  interactive  instruction  makes  it  an  effective  instructional  delivery 
medium.  As  wc  continually  update  technology  courses  to  address  the  innovations,  the  Web  will  increasingly  be 
used  as  an  effective  alternative  delivery  medium.  This  paper  has  presented  the  transformation  of  lecture-based 
course  content  to  computer-mediated  instruction  on  the  World  Wide  Web.  We  welcome  you  to  visit  the  visual 
literacy  Web  site  at  http://www.educ.kcnt.edu/vlo  and  to  share  your  comments  and  suggestions  with  us. 


BEST  COPY  AVAILABLE 

443 


452 


References 

Foster,  D.  F.  & Olsen,  J.  B.  (1997).  “Test  development  and  psychometric  characteristics  of  alternative 
computerized  item  types.”  Roundtable  presented  at  the  American  Educational  Research  Association  annual  meeting, 
Chicago,  IL,  March. 

Grabowski,  B.,  & Pearson,  R.  (1988).  The  development  of  a research  agenda  and  generic  disc  for 
computer-based  interactive  video.  Paper  presented  at  the  Association  for  Educational  Communications  and 
Technology  National  Convention,  New  Orleans,  LA. 

Hannafin,  M.  J.  (1992).  Emerging  Technologies,  ISD,  and  learning  environments:  Critical  perspectives. 

Educational  Technology  Research  and  Development , 40(  1),  49-63. 

Heller,  R.  S.  (1990).  The  role  of  hypermedia  in  education:  A look  at  the  research  issues.  Journal  of 
Research  on  Computing  in  Education , 43,  43 1 -44 1 . 

Heinich,  R.,  Molenda,  M.  Russell,  J.  D.,  & Smaldino,  S.  E.  (1996).  Instructional  media  and  technologies 
for  learning.  Columbus,  OH:  Merrill. 

Kinzic,  M.  B.,  & Bcrdel,  R.  L.  (1990).  Design  and  use  of  hypermedia  systems.  Educational  Technology 
Research  & Development , j>7(2),  5-14. 

Kovalik,  C.  L.  (1994).  Displays  and  communication:  A self-instructional  module.  Unpublished  manuscript. 

Kovalik,  C.  L.  & Dalton,  D.  W.  (1997).  A conceptual  framework  for  assessment:  The  Process/Outcomc 
evaluation  model.  In  O.  Abel,  N.  J Maushak,  & K.  E.  Wright,  (Eds.),  Nineteenth  Annual  Proceedings  of  the 
Association  for  Educational  Communications  and  Technology , 161-168. 

Kovalik,  C.  L.  & Shoffner,  M.  B.  (1997).  Instructional  design  report  for  the  visual  literacy  component  of 
ITEC  27400.  Principles  of  Instructional  Media.  Unpublished  manuscript. 

Kozma,  R.  (1991).  Learning  with  media.  Review  of  Educational  Research,  6/(2),  179-21 1. 

Lamb,  A.  (1996).  Building  treehouses  for  learning:  Technology  in  today’s  classroom.  Emporia,  KS:  Vision 
to  Action. 

Moore,  D.  M.  & Dwyer,  F.  M.  (Eds.).  (1994).  Visual  literacy:  A spectrum  of  visual  learning.  Englewood 
Cliffs,  NJ:  Educational  Technology  Publications. 

Moore,  D.  M„  Myers,  R.  J.,  and  Burton,  J.  K.  (1993).  Multimedia:  Promise,  reality  and  future.  In  Visual 
Literacy  in  the  Digital  Age:  Selected  Readings  from  the  Annual  Conference  of  the  International  Visual  Literacy 
Association,  Rochester,  New  York.  ERIC  Document  Reproduction  Services  No.  ED  370  544. 

Reeves,  T.  (1986).  Research  and  evaluation  models  for  the  study  of  interactive  video.  Journal  of 
Computer-Based  Instruction,  13(4),  102-106. 

Shoffner,  M.  B.  (1997).  Effects  of  instructional  strategics  on  emerging  technology-based  visual  literacy 
instruction:  Problem-based  learning,  networked  hypermedia-based  instruction,  and  cooperative  learning  strategies. 
Unpublished  doctoral  dissertation.  Kent  State  University:  Kent,  OH. 

Shoffner,  M.  B.  & Dalton,  D.  W.  (1998).  Effects  of  problem-based,  networked  hypermedia,  and 
cooperative  strategies  on  visual  literacy  instruction.  In  M.  R.  Simonson  (Ed.),  Proceedings  of  the  1998  Convention 
of  the  Association  for  Educational  Communications  and  Technology.  St.  Louis,  MO:  Association  for  Educational 
Communications  and  Technology. 

Slee,  E.  J.  (1989).  A review  of  the  research  on  interactive  video.  Paper  presented  at  the  Association  for 
Educational  Communications  and  Technology  National  Convention,  Dallas,  TX. 

Wileman,  R.  E.  (1993).  Visual  communicating.  Englewood  Cliffs,  NJ:  Educational  Technology 
Publications  303-3083 


453 


University  Faculty  Use  of  the  Internet 


Yu-mei  Wang 
Arlene  Cohen 
University  of  Guam 


The  Internet,  one  of  the  most  powerful  tools  ever  invented  in  human  history,  has  changed  both  the  way 
people  communicate  with  each  other,  and  the  way  information  is  accessed.  The  rich  resources  on  the  Internet  are 
beneficial  to  all  educational  endeavors,  supporting  teaching,  research  and  other  academic  activities.  Universities  and 
colleges,  where  academic  dialogue  and  information  resources  are  essential  for  professional  success,  are  without  a 
doubt  the  most  likely  to  reap  the  benefits  of  the  Internet. 

Pioneers  on  university  and  college  campuses  first  started  experimenting  with  the  Internet  in  teaching  and 
research,  using  mainly  e-mail,  real-time  conferences,  and  most  recently,  the  World  Wide  Web.  These  experiments 
established  the  value  of  the  Internet  as  an  effective  tool  in  facilitating  teaching  and  research.  However,  the  real 
impact  of  the  Internet  on  university  campuses  will  not  be  effectively  realized  unless  the  majority  of  faculty  begin 
using  the  network. 

Although  the  benefit  of  the  Internet  to  education  is  obvious,  research  investigating  the  use  of  the  variety  of 
Internet  services  by  faculty  on  university  campuses  is  lacking.  A careful  search  of  the  literature  reveals  little 
published  research  on  faculty  use  of  the  Internet.  The  few  available  studies  tended  to  focus  on  the  use  of  a particular 
Internet  service,  such  as  e-mail  or  mailing  lists,  although  the  Internet  provides  many  more  services.  These  services 
include  FTP  (File  Transfer  Protocol),  Telnet,  Gopher  and  the  World  Wide  Web  (WWW).  These  services  can  be 
utilized  as  valuable  educational  resources.  Consequently,  it  is  essential  to  investigate  how  faculty  utilize  the  variety 
of  internet  services.  Only  then,  can  appropriate  strategies  be  developed  to  promote  the  maximum  use  of  the  Internet 
among  faculty. 

The  previous  research  attempted  to  provide  baseline  data  about  the  use  of  the  Internet  by  academics. 
However,  subjects  involved  in  these  studies  were  cither  university  students  or  selective  network  users,  and  limited 
Internet  services  were  examined.  None  of  the  research  dealt  with  all  faculty  within  a university  campus  using  the 
broad  range  of  Internet  resources, 

Schacfcrmcyer  and  Sewell  (1988)  conducted  a survey  via  B1TNET , the  largest  academic  computer  network 
in  the  world.  Their  investigation  focused  on  the  use  of  e-mail  by  BITNET  scholars  to  determine  if  e-mail  was  an 
effective  tool  for  their  professional  development.  The  survey  found  that  most  professionals  used  e-mail  to 
communicate  with  those  who  shared  similar  interests  at  distant  locations.  Users  tended  to  use  e-mail  as  a tool  for 
research  more  than  teaching.  This  finding  was  probably  due  to  the  fact  that  the  majority  of  users  did  not  hold  a 
teaching  job.  For  those  that  did  teach,  access  to  the  Internet  was  limited  for  their  students.  They  also  found  that 
users  who  responded  to  their  survey  were  those  with  quick  and  easy  access  to  the  BITNET  network.  The  study 
identified  "speed*',  "convenience",  "time-saving",  and  "asynchronicity"  as  the  advantages  of  using  e-mail. 

Realizing  the  importance  of  a computer  network  for  academic  development,  Grabowski  and  Pusch  (1990) 
conducted  a study  on  how  graduate  students  at  their  university  used  e-mail.  Their  research  generated  a profile  of 
likely  users  in  a graduate  school  and  studied  factors  that  promote  or  hinder  e-mail  use.  Generally  speaking,  they 
found  that  e-mail  is  used  for  exchanging  information,  asking  questions,  discussing  opinions,  helping  on  course 
assignments,  making  friends,  and  alleviating  boredom.  The  study  also  identified  the  reasons  for  using  e-mail  as  : 
(a)  no  time/place  limits,  (b)  ea.  '>  access,  (c)  knowledge  of  computers,  (d)  case  of  use,  (e)  convenience,  and  (f)  try 
new  things  (p.  280). 

"Mailing  lists"  is  another  service  provided  by  the  Internet.  The  "mailing  list"  concept  has  been  referred  to 
by  various  names  such  as  listservs,  electronic  discussion  groups,  or  c-confcrenccs.  A mailing  list  differs  from  e-mail 
in  that  it  provides  a forum  for  professionals  to  broadcast  their  ideas  to  a larger  audience  with  similar  interests.  Judith 
Wccdman’s  study  (1994)  investigated  how  humanities  scholars  used  a mailing  list  called  Humanist . Her  research 
confirmed  that  the  list  provided  humanities  scholars  with  an  important  vehicle  for  engaging  in  exploratory 
discussions,  seeking  specific  information,  posting  pre-publication  ideas  and  following  new  developments  in  their 
own  and  related  fields.  The  study  revealed  that  humanities  scholars  used  the  list  most  for  research,  followed  by 
professional/technical  work,  with  teaching  being  the  least.  Humanities  scholars  considered  the  list  a useful  resource. 


The  list  allowed  scholars  to  reach  out  to  a larger  portion  of  the  community  of  scholars,  not  previously  available 
andthe  majority  of  them  preserved  messages  posted  on  the  list  for  a long  time. 

In  another  study  involving  mailing  lists,  Kovas  (1995)  confirmed  that  mailing  lists  are  a valuable  internet 
tool  for  academics.  The  study  examined  the  use  of  mailing  lists  by  professionals  in  the  Library  and  Information 
Science  field.  The  survey  was  sent  to  57  library  and  information  science  related  mailing  lists.  The  findings  of  the 
study  indicated  that  mailing  lists  are  an  important  information  source  for  these  professionals,  both  for  personal  use 
and  job-related  service.  They  used  mailing  lists  as  a source  for  their  own  professional  and  research  development, 
while  at  the  same  time  using  the  lists  to  enhance  the  quality  of  library  services  to  patrons.  Mailing  lists  were  used  as 
a complement  to  other  information  sources  such  as  academic  journals,  face-to-face  conferences,  regular  mail  and 
phone  contacts. 

Landis'  study  (1995)  focused  on  how  science  educators  used  a variety  of  internet  resources.  The  survey 
was  posted  to  27  newsgroups  and  a small  group  of  science  educators  in  secondary  schools  and  colleges  responded. 
The  results  showed  that  these  science  teachers  used  the  internet  for  activities,  which  included  communicating  by  e- 
mail,  conducting  library  searches,  downloading  information,  and  participating  in  newsgroups.  Respondents, 
including  the  most  experienced  users,  reported  difficulties  in  downloading  information  from  the  Internet. 

An  Australian  study  (1994)  looked  at  how  academics  in  Australia  universities  use  the  Australian 
Academic  and  Research  Network  (AARNet)  to  support  their  academic  role.  Technology  directors  from  thirteen 
universities  were  each  asked  to  identify  twenty  regular  users  of  AARNet  in  their  institutions.  A survey  was  then 
mailed  to  these  AARNet  users.  The  results  showed  that  e-mail  was  the  service  most  used  (99%)  and  considered 
most  important  (66%),  followed  by  remote  login  (85%),  News  (44%),  Gopher  (15%)  and  WAIS  (6%). 

The  study  provided  strong  evidence  that  Australian  academics  used  AARNet  "to  enhance  the  efficiency, 
quality  and  productivity  of  their  academic  work"  (p.  29).  Australian  academics  were  utilizing  AARNet  to  facilitate 
scholarly  activity  in  their  field;  seek  and  disseminate  information,  exchange  ideas  and  facilitate  professional 
collaboration.  Respondents  perceived  AARNet  beneficial  in  terms  of  "increased  efficiency;  improved  access  to 
facilities,  data,  and  collaborators;  and  an  enhanced  sense  of  disciplinary  collegiality"  (p.  29). 

Homey  and  Henriquez  (1993)  conducted  the  first  systematic  and  large  scale  survey  on  educators1  use  of  the 
Internet.  The  study  involved  550  educators,  and  found  that  the  network  is  becoming  an  important  component  in 
their  teaching  and  learning  activities.  However,  the  survey  focused  on  K-12  educators  and  did  not  give  any 
indicator  of  how  university  faculty  used  the  Internet. 

To  date,  no  research  has  been  found  focusing  on  the  university  faculty's  use  of  a variety  of  internet  services 
in  the  United  States.  This  exploratory  study  was  designed  to  investigate  the  use  of  services  available  on  the  Internet 
by  faculty  in  a public  university  in  the  United  States;  the  manner  and  frequency  of  faculty  use  of  the  internet;  their 
perception  of  the  role  of  different  Internet  services  in  support  of  teaching  and  research;  and  the  factors  associated 
with  their  use  of  these  internet  services. 

Methodology 

The  sample  population  consisted  of  all  the  faculty  in  a public  university  in  the  United  Slates  during  the 
spring  semester,  1996.  The  total  sample  population  for  this  study  was  180.  A questionnaire  was  developed 
containing  items  related  to  each  of  the  research  questions.  The  questionnaire  contained  30  questions  with  yes/no, 
multiple  choice,  and  open-ended  questions.  The  questionnaire  was  distributed  at  faculty  meetings.  One  hundred 
and  fifty-eight  completed  questionnaires  were  collected.  The  response  rate  was  88%. 

Results 

Background  information  on  the  faculty  included  gender,  age,  their  internet  experience,  and  the  way  they 
accessed  the  Internet . Approximately  tv/o  thirds  of  the  faculty  responding  to  the  survey  were  males  (66%).  Faculty 
between  the  ages  of  41-50  comprised  the  largest  age  group  (45%).  Eighty-five  percent  of  the  faculty  responding  to 
the  survey  were  using  the  Internet,  with  their  internet  experience  varying  from  months  to  years.  E-mail  is  the 
service  most  used  by  the  faculty  (96%),  followed  by  the  World  Wide  Web  (55%),  Gopher  (48%-),  Mailing  Lists 
(29%),  and  FTP  (22%9-  Approximately  half  of  the  faculty  had  internet  access  at  home  (Figure  1 & Figure  2). 
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Figure  I.  Gender,  Age  and  Internet  User 


Figure  2.  Use  of  Internet  Services,  Access  and  Internet  Experience 
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The  time  faculty  spent  using  internet  services  varied  considerably  (Table  1).  Approximately  53%  of  the 
faculty  spent  over  three  hours  per  week  using  e-mail,  as  opposed  to  just  25%  using  mailing  lists  over  3 hours  on  a 
weekly  basis.  Additionally,  21%  used  WWW,  1 1 % used  Gopher,  and  only  4%  used  FTP  over  3 hours  per  week. 
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Table  L Time  Spent  on  the  Use  of  the  Internet  Services  ( Hours  Per  Week) 


X hour 

2 hours 

3 hours 

4 hours 

5 hours 
and  over 

n 

E-mail 

22% 

25% 

13% 

17% 

23% 

128 

Mailing  List 

39% 

36% 

0% 

6% 

19% 

31 

WWW 

53% 

26% 

7% 

5% 

9% 

62 

Gopher 

72% 

17% 

4% 

2% 

5% 

53 

FTP 

91% 

5% 

4% 

0% 

0% 

23 

The  reasons  faculty  use  the  variety  of  internet  services  are  presented  in  Figure  4.  The  major  reasons  given 
for  using  e-mail  were  to  communicate  with  professionals  (93%)  and  friends  (75%).  Mailing  lists  wore  used  to 
exchange  experiences  with  other  professionals  (87%),  generate  ideas  for  teaching  (50%)  and  research  (47).  Faculty 
used  WWW,  Gopher  and  FTP  for  research  to  a higher  degree  than  for  teaching  (78%  vs.  56%;  75%  vs.  57%;  69% 
vs.  52%). 


Figure  3.  Reasons  for  Using  Different  Internet  Services 
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For  those  responding  to  the  survey,  only  three  faculty  provided  reasons  for  not  using  e-mail.  They  all 
indicated  they  did  not  have  a need.  A majority  of  the  faculty  did  not  use  mailing  lists.  Almost  99%  reported  they 
did  not  know  how  to  access  lists  appropriate  to  their  field  or  interest  (Table  2).  Faculty  that  did  not  use  the  other 
internet  services  generally  had  an  interest  and  need,  but  lacked  the  knowledge  necessary  to  use  the  service. 

Table  2.  Reasons  for  the  Non-Use  of  the  Different  Internet  Tools 


Lack  of 

Knowledge 

Lack  of  Access 

Lack  of 
Need 

Lack 

Interests 

of  n 

E-mail 

67% 

33% 

100% 

67% 

3 

Mailing  List 

38% 

99% 

10% 

7% 

94 

Gopher 

57% 

13% 

6% 

7% 

40 

WWW 

62% 

27% 

5% 

2% 

37 

FTP 

57% 

10% 

8% 

5% 

59 

When  surveyed  about  their  perception  of  the  importance  of  internet  tools  to  support  teaching  and  research, 
the  percentages  considering  internet  tools  as  important  or  very  important  were  78%  fore-mail,  63%  for  mailing  lists, 
69%  for  Gopher,  74%  for  the  World  Wide  Web,  and  60%  for  FTP  (Table  3). 
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Table  3.  Perception  of  importance  on  the  Internet  Tools  in  Teaching  and  Research 


Not  important 

Somewhat 

Important 

Important 

Very  Important 

n 

E-mail 

5% 

17% 

25% 

53% 

139 

Mailing 

List 

10% 

27% 

30% 

33% 

90 

Gopher 

7% 

24% 

38% 

31% 

95 

WWW 

6% 

20% 

35% 

39% 

105 

FTP 

13% 

27% 

32% 

28% 

68 

There  was  no  difference  in  the  use  of  e-mail  between  the  male  and  female  faculty,  although  there  was  a 
slightly  higher  percentage  of  female  faculty  who  used  mailing  lists  than  did  male  faculty.  More  male  faculty  used 
Gopher,  WWW  and  FTP  than  female  faculty.  However,  due  to  the  imbalance  of  numbers  between  male  (66%)and 


female  (34%),  a conclusion  cannot  be  made. 

Only  four  faculty  members  in  the  over  60  age  group  responded  to  the  survey.  Faculty  in  the  31-40  age 
group  were  the  most  active  in  using  a variety  of  the  internet  tools.  There  was  a higher  percentage  of  use  of  each 
internet  service  in  this  age  group  than  there  was  in  other  age  groups.  Again,  due  to  the  imbalance  of  numbers  in  age 
groups,  the  relationship  between  age  and  use  of  internet  services  can  not  be  determined. 

Table  4.  Effect  of  Gender  and  Age  on  the  Use  of  the  Internet  Tools 

E-Mail 

Mail  List 

Gopher 

WWW 

FTP 

n 

Gender 

Male 

98% 

28% 

53% 

60% 

26% 

88 

Female 

98% 

32% 

38% 

46% 

14% 

51 

Age 

30-40 

100% 

47% 

50% 

59% 

31% 

32 

41-50 

99% 

24% 

44%. 

52% 

16% 

68 

51-60 

94% 

29% 

47% 

56% 

21% 

34 

Over  60 

100% 

0% 

75% 

75% 

25% 

4 

Faculty's  perception  of  the  role  of  a internet  service  in  support  of  leaching  and  research  was  associated  with 
the  use  of  a particular  service.  There  was  a higher  percentage  of  use  of  an  internet  service  among  those  who 
considered  it  important  or  very  important  in  support  of  teaching  and  research  than  there  was  among  those  who 
thought  otherwise  (Table  5).  Time  spent  on  the  internet  was  related  to  the  use  of  internet  services.  The  longer  the 
user  was  exposed  to  the  Internet,  the  more  likely  the  user  tended  to  utilize  the  variety  of  Internet  services  (Table  6). 
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Table  5.  Effect  of  Faculty's  Perception  on  the  Use  of  the  Internet  Tools 


Not  Important 

Somewhat 

Important 

Important 

Very  Important 

n 

E-mail 

" 80 % 

95% 

100% 

~99%”~ 

129 

Mailing  List 

0% 

32% 

46% 

60% 

82 

Gopher 

17% 

61% 

72% 

79% 

84 

WWW 

40% 

61% 

67% 

80% 

96 

FTP 

0% 

54% 

37% 

63% 

60 

Table  6.  Effect  of  Faculty's  Internet  Experience  on  the  Use  of  the  Internet  Tools 


1-6  Month 

7-12  Month 

1 Year 

2 Years 

3 Years 

Over 

Years 

3 n 

E-mail 

96% 

95% 

100% 

100% 

100% 

100% 

140 

Mailing 

18% 

5% 

31% 

30% 

41% 

61% 

140 

List 

WWW 

18% 

50% 

50% 

56% 

47% 

61% 

140 

Gopher 

18% 

55% 

58% 

63% 

59% 

72% 

140 

FTP 

0% 

0% 

19% 

41% 

18% 

56% 

140 

Discussion 

This  study  shows  that  a majority  of  the  faculty  at  the  surveyed  university  were  exposed  to  internet  use. 
Eighty-five  percent  of  the  faculty  used  at  least  one  of  the  internet  services.  Faculty  were  aware  of  the  roie  of  the 
Internet  in  their  professional  development.  They  used  the  internet  in  support  of  teaching  and  research,  although 
some  of  the  internet  resources  were  utilized  in  a limited  way. 

E-mail  is  still  one  of  the  most  popular  services  among  the  faculty  ' ,i  though  other  internet  services,  such 
as  the  World  Wide  Web,  arc  gaining  popularity  in  recent  years.  The  popularity  of  e-mail  is  probably  due  to  e-mail's 
role  in  allowing  quick  id  direct  personal  communication.  Faculty  judged  e-mail  an  exciting  and  convenient  tool  to 
keep  them  connected  • ith  professionals  world  wide.  They  reported  a variety  of  academic  activities  via  c-mail. 
Since  other  internet  services  are  still  new  to  most  faculty,  this  might  be  another  reason  for  high  c-mail  use.  Faculty 
need  time  to  learn  the  technology. 

Faculty  use  of  mailing  lists  was  low  (29%).  Mailing  lists  provide  an  important  forum  for  professionals  to 
exchange  ideas,  information  and  experiences.  They  also  allow  for  frequent  dialogues  and  heated  debates  among 
professionals,  contributing  greatly  to  teaching  and  research.  Faculty  who  participated  in  mailing  lists  were  highly 
positive  about  their  experiences.  These  faculty  reported  using  mailing  lists  to  keep  current  on  issues  of  concern  and 
maintain  contact  with  professionals  having  the  same  interests.  One  faculty  wrote:  "When  I was  preparing  for  a new 
course.  I found  four  mailing  lists,  all  of  which  included  a teacher  whose  students  wished  to  communicate  with 
students  from  another  culture.  I benefited.  They  benefited.  We  all  did  what  wc  do  better." 

The  majority  of  the  faculty  surveyed  had  no  knowledge  about  searching  for  and  subscribing  to  an 
appropriate  mailing  list,  which  can  be  a formidable  task.  At  times,  it  can  be  like  seeking  a needle  in  a hay  since 
countless  many  mailing  lists  exist  in  almost  all  disciplines.  When  a seemingly  appropriate  mailing  list  is  found  and 
a user  subscribes,  he/she  might  then  find  that  it  is  not  appropriate  to  his/her  needs.  Further,  the  dynamic  nature  and 
frequent  obsolescence  of  mailing  lists  contribute  to  frustrations  encountered  with  finding  appropriate  mailing  lists. 
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However,  to  alleviate  some  of  the  frustrations,  information  about  mailing  lists  is  becoming  more  pervasive. 
Faculty  can  use  these  information  to  locate  an  appropriate  mailing  list,  rather  than  relying  on  random  database 
search.  Some  academic  journals  and  magazines  now  recommend  mailing  lists  to  their  subscribers.  Faculty  who  find 
beneficial  mailing  lists  can  pass  the  information  to  their  colleagues.  Using  mailing  lists  that  arc  recommended  and 
tried  by  other  professionals  saves  a great  amount  of  time  and  anxiety  for  faculty  new  to  mailing  lists.  Another 
source  of  information  about  mailing  lists  can  be  obtained  from  professional  librarians  on  university  campuses. 

For  information  search  and  retrieval,  the  World  Wide  Web  showed  the  highest  rate  of  use,  with  Gopher 
rated  the  second  and  FTP  the  lowest.  Compared  with  Gopher  and  FTP,  the  World  Wide  Web  has  a better  user- 
interface  for  information  access  and  retrieval.  It  has  the  advantage  of  allowing  users  to  make  links  while  navigating 
through  information.  Users  can  also  see  the  information  before  they  decide  whether  they  want  a copy  of  it.  In 
recent  years,  enormous  amounts  of  information  is  available  on  the  World  Wide  Web.  However,  each  service  has  its 
advantages  and  disadvantages.  Faculty  need  to  be  aware  of  potentials  and  limitations  of  each  internet  service  and 
utilize  them  to  their  benefit. 

The  results  of  the  study  indicate  that  faculty  used  Wo  id  Wide  Web,  Gopher,  and  FTP  more  for  research 
than  teaching.  Moreover,  it  seems  that  faculty  turned  to  other  resources  such  as  e-mail  and  mailing  lists  for 
exchanging  and  generating  teaching  ideas. 

Even  though  the  World  Wide  Web  comprised  the  highest  rate  of  use,  only  about  half  of  the  faculty  were 
using  it  (55%).  Among  those  who  used  the  World  Wide  Web,  about  one  fifth  spent  over  three  hours  per  week  using 
the  Web.  Faculty  expressed  their  frustration  in  their  comments  which  included,  "The  World  Wide  Web  is  short  on 
information,  causing  waste  of  time  searching  different  web  sites,”  "The  World  Wide  Web  is  complex  and  time 
consuming,”  "Get  stuck  in  the  mud  during  surfing.” 

Information  searching  on  the  World  Wide  Web  is  a challenge.  The  way  information  is  arranged  and 
accessed  differs  from  traditional  information  resource  organization  in  a library.  Faculiy  need  to  break  their  habitual 
patterns  of  information  seeking  behavior  and  develop  new  information  seeking  strategies.  At  the  moment, 
information  searching  on  the  World  Wide  Web  for  most  people  is  like  combing  the  beach  for  treasures.  This 
process  of  "treasure  hunting"  is  not  attractive  to  faculty  since  most  of  the  lime,  they  want  immediate  access  to  the 
information  they  need.  If  they  experience  any  frustration  in  accessing  and  retrieving  information,  faculty  will  look 
to  other  sources. 

Information  searching  on  the  World  Wide  Web  is  complex  since  it  requires  both  technical  skill  and 
information  searching  skills.  Neither  skill  alone  can  complete  the  task.  The  types  of  information  available  on  the 
World  Wide  Web  might  also  influence  faculty  use.  One  possible  solution  is  cooperation  between  campus  librarians 
and  faculty  to  develop  their  information  searching  skills.  Librarians  and  faculty  can  work  together  to  locate  and 
disseminate  discipline-specific  World  Wide  Web  sites. 

The  results  of  this  study  pointed  to  the  need  for  faculty  training  in  order  to  promote  the  maximum  use  of 
internet  services.  This  was  also  indicated  in  the  many  comments  found  in  the  survey. 

Training  workshops  should  be  offered  in  a variety  of  formats,  including  hands-on  workshops,  seminars,  and 
strategy  sharing  workshops.  Workshops  should  not  only  focus  on  the  technical  aspect  of  the  internet,  but  on 
increasing  faculty's  awareness  of  available  internet  resources,  the  role  of  the  Internet  in  teaching  and  learning, 
strategics  of  surfing  the  Internet,  and  the  potential  and  limitations  of  each  internet  service.  Workshops  should 
answer  questions  such  as;  What  is  the  appropriate  tool  for  a particular  networking  activity?  What  are  the  strategies 
for  information  searching  and  retrieval?  How  to  incorporate  the  Internet  resources  into  classroom  activities?  How  to 
utilize  resources  such  as  on-line  help  and  Internet  reference  books? 

Training  should  be  offered  on  different  levels;  university,  college,  and  division.  Training  workshops 
should  provide  immediate  benefits  to  faculty  and  be  relevant  to  their  professional  development.  When  faculty  can 
sec  a clear  pay  off,  they  will  be  motivated  to  learn  to  use  the  Internet. 

Training  should  be  done  as  a team  effort.  Librarians,  experienced  internet  faculty  users,  and  technical  staff 
should  work  together  to  design  and  offer  workshops,  utilizing  each  others'  expertise.  With  their  professional  skills 
in  seeking  and  accessing  information,  librarians  can  play  a key  role  in  training,  coupled  with  the  subject  expertise  of 
faculty  and  ihe  "behind  the  scenes"  technical  support  of  the  technology  staff,  training  curriculum  can  be  designed  to 
meet  the  needs  of  faculty  at  all  levels. 

Though  faculty  training  is  essential  in  promoting  the  use  of  the  Internet,  faculty  can  only  gain  experience 
by  actually  using  the  Internet.  Technical  support  is  crucial  to  ensure  faculty's  use  of  the  Internet.  Even  the  most 
experienced  users  can  encounter  difficulties  using  the  Internet.  The  majority  of  the  faculty  reported  that  their  most 
frustrating  experience  was  having  technical  problems  while  using  the  internet.  This  included  logging  on  failure. 
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computer  crashes,  getting  cut  off  in  the  middle  of  sessions,  and  difficulties  in  downloading  and  uploading  files. 
Faculty  need  access  to  technicians  who  are  available  at  any  time  to  address  their  technical  problems.  As  one  faculty 
indicated:  "I  wish  there  were  more  specific  individuals  on  campus  who  could  address  technical  questions  about  the 
Internet." 

It  is  unlikely  that  universities  can  hire  an  adequate  number  of  technicians  to  provide  such  a timely  service. 
Usually,  available  technical  staff  on  campus  arc  busy  operating  and  maintaining  the  campus  network,  making  it 
almost  impossible  to  help  individual  faculty  solve  his/her  technical  problems.  One  solution  might  be  to  utilize 
students  as  resources.  For  example,  students  majoring  in  the  Computer  Science  could  take  practicums  and  be 
assigned  to  specific  departments  or  colleges  to  help  faculty  members. 

i inclusions 

This  study  generated  baseline  information  and  provided  insights  on  how  faculty  use  the  internet.  With  the 
aggressive  development  of  computer  technology,  the  Internet  will  play  an  even  greater  role  in  academic  settings, 
presenting  a great  challenge  to  university  faculty.  There  are  many  factors  that  impact  faculty  use  of  the  internet.  It 
is  essential  to  continue  studying  how  faculty  use  the  Internet  and  identify  the  factors  associated  with  internet  use  so 
that  the  benefit  of  the  internet  can  be  fully  realized. 
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Abstract 

This  study  examines  the  efficacy  of  a class  discussion  conducted  by  listserv  which  was  used  instead  of 
classroom  meetings  for  a graduate  seminar.  Advances  in  communication  have  stimulated  educators  to  evaluate  new 
methods  of  instruction  taking  advantage  of  the  growing  interest  in  distance  learning. 

Educators  have  recognized  that  the  advent  of  computers  and  the  World  Wide  Web  is  profoundly  changing 
the  methods  used  in  education  (Goodman,  1995;  Polyson,  Saltzberg,  Goodwin-Jones  1996;  Boon,  1996).  One 
aspect  of  this  evolving  field  is  the  use  of  lislservcs  for  distance  learning  and  specifically  its  use  as  a means  of 
distance  classroom  interaction  (Partec,  1996). 

The  impetus  to  use  e-mail  for  educational  purposes  is  derived  from  the  many  possible  advantages  it  offers. 
Theoretically,  e-mail,  in  a distance  learning  environment,  should  allow  a means  for  interaction  among  diverse 
students  and  teachers  that  would  otherwise  be  missed  in  a classroom  situation.  Also,  by  overcoming  the  problems  of 
distance  and  time,  e-mail  may  allow  students  from  great  distances  and  other  cultures  to  participate  in  the  class,  and 
consequently  to  enhance  the  value  of  the  course  for  all  the  participants.  Furthermore,  e-mail  may  allow  interaction 
at  the  convenience  of  the  participants  (Partee,  1996).  Another  advantage  of  e-mail  is  the  time  it  allows  the  user  to 
construct  responses  that  arc  well  thought  out  and  not  lime  limited  as  in  traditional  classroom  seminars.  With  the 
increasing  use  of  c-mail  in  education,  it  is  important  to  make  the  best  use  of  this  modality.  Having  this  objective  in 
mind,  the  focus  of  this  study  was  to  evaluate  the  communication  that  transpired  during  a graduate  seminar  conducted 
by  listserv.  This  study  was  exploratory  in  that  the  goal  of  the  study  was  “to  investigate  a little  understood 
phenomena”  (Marshall  and  Rossman,  1995,  p.  41)  and  to  identify  strengths  and  weaknesses  in  order  to  “generate 
hypotheses  for  further  research”  (Marshall  and  Rossman,  1995,  p.41 ).  It  is  not  a definitive  study. 

Review  of  the  literature 

While  there  is  increasing  information  about  the  use  of  telecommunication  in  learning  and  the  use  of  the 
Internet  for  communication  and  discussion  groups,  there  is  very  little  information  on  the  actual  use  of  e-mail  in 
conducting  a class  by  listserv.  One  study  completed  at  Queens  University  in  Belfast  compared  computer-supported 
seminars  with  facc-to-facc  seminars  (Newman,  Johnson,  & Webb,  1996).  They  found  that  critical  thinking  was 
deeper  in  computer  conferences.  The  evidence  of  critical  thinking  was  demonstrated  by  the  cognitive  reasoning 
skills.  The  Belfast  study  group  developed  a set  of  paired  opposites,  one  an  indicator  of  surface  processing  and  one 
of  in-depth  processing.  “For  example,  making  judgments  without  offering  justification,  versus  setting  out  the 
advantages  and  disadvantages  of  a situation  or  solution.”  (Newman,  1996,  p.66).  They  also  found  that  the  students 
in  the  computer  conferences  brought  in  more  outside  relevant  material  and  were  hotter  at  linking  together  ideas  and 
solutions.  On  the  other  hand,  the  facc-to-facc  seminar  students  came  up  with  new  ideas  more  often.  The  c-mail 
students  seem  to  have  a more  serious  style  and  produced  a higher  ratio  of  important  statements. 

Another  paper  described  the  results  of  a conference  for  teachers  in  which  interactive  e-mail  discussions 
were  conducted  over  a period  of  2 months  prior  to  a 3 day  seminar  and  for  1 month  afterwards  (Rud,  1995).  The 
participants  felt  that  the  use  of  computer  communication  allowed  them  to  explore  their  own  practice  as  a teacher 
more  by  fostering  awareness  of  wider  issues  in  information  technology.  It  also  brought  to  attention  multiple 
avenues  for  effectiveness  in  teaching  many  kinds  of  students.  Furthermore,  it  allowed  them  to  share  ideas  with 
others  who  lived  at  a distance  from  them. 

A study  of  communication  patterns  among  undergraduate  student  groups  using  c-mail  was  done  at  the 
University  of  Minnesota  by  Miller  (1994).  The  study  was  of  exploratory  nature,  but  it  did  demonstrate  the  ability  to 
manage  micro-level  communication  dynamics  in  the  transcripts.  Another  report  on  the  use  of  a listserv  in  a 
communication  course  at  a liberal  art  college  concluded  that  e-mail  afforded  a valuable  additional  channel  of 
communication  between  instructor  and  students  (Bruning,  1995).  A report  on  the  use  of  a listserv  for  a graduate 
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elementary  education  course  concluded  that  the  listscrv  enhanced  both  the  graduate  and  teaching  experience  of  the 
teachers  while  providing  an  alternative  communication  option  (McGinnis,  1995). 

A study  by  Ross  (1995),  provided  four  recommendations  for  using  a listserv. 

1 . Technical  factors  must  be  addressed  to  insure  easy  confident  communication. 

2.  Novice  users  need  skill  development  with  the  technology.  This  skill  can  be  a prerequisite  for  the  class 
or  training  in  e-mail  becomes  part  of  the  course  content. 

3.  Arrangements  for  setting  up  students'  accounts  should  be  made  in  advance. 

4.  Students  should  be  informed  early  on  about  the  expectations  of  their  participation  on  the  listserv. 

Because  of  the  growing  use  of  e-mail  communication  it  is  important  for  educators  to  learn  how  best  to  use 

this  new  method  of  instruction.  It  is  necessary  for  teachers  to  learn  from  the  experience  of  those  already  using  this 
method  so  that  they  may  better  formulate  their  own  teaching  program. 

Research  Questions 

The  purpose  of  this  study  was  to  examine  what  kind  of  communication  transpired  during  a graduate 
seminar  in  which  the  discussion  was  conducted  by  listserv.  Two  questions  were  of  primary  interest  in  this  study. 
First,  was  this  mode  of  communication  successful  for  the  purpose  of  the  course?  The  purpose  of  the  course  was  for 
each  student  to  demonstrate  the  ability  to  critically  analyze  and  synthesize  theoretical  and  conceptual  works  from  the 
Field  of  instructional  design  and  technology.  To  answer  this  question,  these  questions  were  asked,  . 

9.  Was  there  evidence  of  critical  thinking  by  the  students  revealed  in  the  e-mail  communications? 

10.  Did  the  students  correctly  understand  the  previous  communications? 

1 1 . Were  any  misunderstandings  corrected? 

12.  Were  emotions  revealed  in  the  communications? 

The  second  question  was  how  could  this  mode  of  communication  be  improved  as  a means  for  replacing  or 
supplementing  face-to-face  classroom  discussion? 

Methodology 

Participants 

Eight  students  and  two  faculty  members  made  up  the  discussion  group.  Of  the  students,  five  were  female 
and  three  were  male.  There  was  one  female  and  one  male  faculty  member.  Six  students  were  working  on  their 
Doctorate  of  Education  degree,  and  two  students  were  working  on  their  Master  of  Science  degree.  All  the  students 
had  a concentration  in  ID&T.  Students  could  use  cither  a free  University  c-mail  account  or  any  e-mail  account  of 
their  choosing. 

Design  of  the  study 

This  is  a retrospective  study  examining  all  of  the  listserv  messages  that  occurred  during  the  seminar.  The 
participants,  both  students  and  faculty,  were  unaware  that  this  study  would  be  conducted.  The  listserv  software 
automatically  archived  all  c-mail  messages  for  future  use.  In  addition  to  study  of  the  listserv  messages,  all  the 
students  were  asked  to  complete  a questionnaire  a the  end  of  the  course.  The  instructors  were  interviewed 
concerning  their  impressions  about  the  use  of  listserv  for  the  discussion  in  this  class. 

The  class  chosen  for  this  study  was  a seminar  in  Instructional  Design  and  Technology:  Perspectives  Past, 
Present,  Future  led  by  Dr.  Gary  Morrison  and  Dr.  Deborah  Lowther  at  the  University  of  Memphis.  The  class  met 
twice  face-to-face  at  the  beginning  of  the  semester  and  once  at  the  end.  Other  than  these  three  meetings,  the  class 
was  conducted  by  listserv.  Twenty  percent  of  the  student’s  grade  was  based  on  their  participation  in  the  class 
discussion. 

This  class  was  chosen  for  analysis  because  it  was  one  of  the  first  efforts  using  listserv  as  a means  of  class 
discussion  at  the  University  of  Memphis.  In  many  ways  this  course  provided  an  ideal  subject  for  analysis  to  answer 
the  questions  proposed  by  the  research  inquiry.  The  course  had  a small  number  of  students,  so  everyone  had  ample 
opportunity  to  participate  in  the  discussion.  The  students  and  instructors  were  all  computer  competent.  Another 
advantage  in  the  choice  of  this  course  was  that  the  discussion  played  a significant  role  in  developing  the  purpose  of 
the  course  and  leant  itself  well  to  the  use  of  listserv. 

Procedure 

A listserv  is  a mailing  list  of  collections  of  c-mail  messages  that  pertain  to  specific  topics  of  interest.  The 
c-mail  messages  arc  collected  at  one  site,  also  called  a listserv,  and  mass  c-mailcd  to  those  who  subscribe  to  the 


particular  list  (Settles,  1995).  All  the  posts  that  were  generated  as  part  of  this  course  were  archived  and  used  as  the 
basis  of  this  study.  Because  of  the  nature  of  the  research  questions  and  the  data  to  be  analyzed,  it  was  decided  to  use 
a qualitative  analysis  approach  for  this  study. 

The  data,  questionnaires,  and  interview  transcripts  were  analyzed  by  one  researcher  and  verified  by  a 
second  reviewer.  Where  disagreement  occurred  a consensus  was  achieved  through  discussion. 

Where  names  were  used  in  the  quoted  communication,  names  were  changed  to  protect  the  anonymity  of  the 
participant,  but  the  gender  of  the  participant  remained  the  same. 

The  first  step  in  analyzing  the  communications  w rs  to  count  the  total  number  of  communications  that 
occurred  as  part  of  the  class  discussion  during  the  eight  wee.  ■.ummer  session.  Then,  the  number  of  communications 
by  each  instructor  and  each  student  were  totaled.  The  messages  were  then  analyzed  by  the  thematic  content  analysis 
method  (Marshall,  & Rossman,  1995).  Question  #1  was  answered  by  using  a subset  of  questions,  a-c,  answered 
“yes”  or  “no”  and  using  all  the  posts  of  the  students  as  the  basis  of  decision.  These  questions  have  been  outlined  by 
Mason  (1991 ) to  evaluate  critical  thinking  in  computer  communication. 

26.  “Did  the  participants  make  reference  to  previous  messages?” 

27.  “Did  they  draw  on  their  own  experience?” 

28.  “Did  they  refer  to  course  materials?” 

29.  “Did  they  refer  to  relevant  materials  outside  the  course?” 

30.  “Did  they  initiate  new  ideas  for  discussion?” 

(pg-  H4). 

To  answer  question  #2,  each  communication  was  classified  as  “yes”  or  “no.”  The  answer  to  question  #3 
will  be  in  narrative  form  as  analysis  if  the  communication  indicates  there  was  misunderstanding.  For  question  #4 
the  instances  of  emotion  were  enumerated,  categorized,  and  evaluated. 

The  definition  of  humor  used  in  the  context  of  computer  mediated  communication  (CMC)  follows  the 
criteria  of  Chiaro,  Hymes,  Oring,  and  Palmer.  Humor  occurs  when  there  is  a sudden  incongruity  or  transgresses 
normalcy  (Palmer,  1992).  This  is  dependent  on  the  surrounding  linguistic,  geographical,  sociocultural,  and  personal 
boundaries  of  the  people  for  whom  the  humor  is  intended  (Chiaro,  1992,  Oring,  1992,  and  Palmer,  1994).  The 
framing  of  humor  is  also  important.  A wink,  a gesture,  or  musical  accompaniment  are  examples  of  cues  that  are 
sometimes  used  in  face-to-face  communication  (Hymes,  1994).  In  CMC  these  cues  are  more  limited,  but  included 
the  use  of  emoticons  or  wording  to  point  out  that  something  is  meant  to  be  funny. 

After  tabulation  of  the  results  of  questions  #1  through  #4,  the  investigator  sought  to  answer  the  following 
questions  to  complete  the  answers  to  the  first  research  question. 

• Did  the  on-line  communication  contribute  significantly  to  the  value  of  the  course? 

• What  improvements  could  be  made  in  the  on-line  discussion? 

• What  were  the  strengths  of  this  type  of  communication? 

• What  were  the  weaknesses? 

• Did  students  change  or  broaden  their  views? 

A questionnaire  with  13  questions  was  sent  to  each  student  participant  to  assist  in  answering  these  questions.  Both 
faculty  members  were  interviewed  as  part  of  the  study. 

Results 

Analysis  of  e-mail  messages 

The  first  analysis  was  to  determine  the  extent  of  critical  thinking  by  the  students.  Out  of  a total  number  of 
464  communications  by  the  students,  the  vast  majority  referred  back  to  a previous  message.  One  student  said,  “I  was 
surprised  by  the  lack  of  agreement  or.  a definition  of  ID  in  our  seminar  group.  1 expected  the  article  authors  to 
disagree,  but  I was  not  prepared  for  the  diversity  of  opinion  among  our  class  members.”  Another  example,  “In 
response  to  Joanne’s  question,  computer....”  There  were  only  seven  instances  when  the  students  failed  to  refer  to 
another  message.  An  example  of  this  was,  “Sorry  other  ID  Seminar  people,  but  I needed  to  reach  George  and  his  e- 
mail  address  is  not  working.  You  can  disregard  this  message.” 

Furthermore,  there  were  numerous  (40)  references  to  the  student’s  own  experience.  For  example,  one 
student  stated,  “Wc  did  an  informal  survey  in  one  of  my  classes  several  years  ago.”  Another  student  in  another 
context  related  that,  “I  have  done  something  like  this  in  the  classroom.”  The  students  also  made  frequent  reference 
(41)  to  the  course  reading  material.  An  example  of  this  was,  “I  honestly  felt  that  the  two  Heinich  articles 
contributed  little  to  defining  the  ID  field,”  and  “the  Merrill  article  really  annoyed  me  because...”  There  was  also 
reference  made  to  relevant  material  outside  the  course  reading  (17).  For  example,  “There  was  an  article  many  years 
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ago  (I  have  the  reference  somewhere),  that  stated...”  In  other  instances,  title  and  author  of  the  references  were 
given.  One  student  said,  “I  found  a paper  at  this  WWW  site:  http:// 1 29.7.1 60.78/intro. html.  I thought  this  was 
appropriate  given  our  discussion  last  night.”  New  ideas  were  initiated  in  14  of  the  communications.  An  example  of 
a new  idea  and  a reference  to  reading  outside  the  course  reading  was,  “In  discussing  the  nature  of  concept  learning 
(Love  and  Snow)  what  role  does  primary  experience  and  representational  experience  play  in  conceptual 
understanding?  How  do  infants  begin  to  conceptualize?  Is  there  a transition  from  a behaviorist  learning  model  to  a 
constructivist  learning  model?” 

It  is  clear  from  study  of  the  communications  that  the  answers  to  the  five  sub  questions  under  question  one 
are  all  yes.  The  evidence  for  critical  thinking  as  manifest  by  the  listserv  communications  was  very  strong  as  defined 
by  Mason  (1991). 

Questions  two  and  three  were  intended  to  discover  if  misunderstandings  occurred  as  a result  of  using 
listserv  for  the  group  discussion  instead  of  being  together  in  the  same  room.  There  were  ten  instances  in  the 
communications  that  indicated  a lack  of  understanding  of  a previous  message.  It  is  likely  that  all  of  these  were 
misunderstandings  that  could  easily  have  occurred  in  face-to-face  discussions  and  were  not  attributable  to  the 
medium  of  communication  itself.  In  each  case  these  misunderstanding  were  easily  and  quickly  corrected  by  an 
ensuing  communication.  An  example  of  was,  “In  response  to  all  of  this,  I actually  did  not  mean  the  physical 
environment  as  much,  rather  the  myriad  of  activities  (lectures,  homework,  classwork)  which  composes  the  learning 
environment.” 

One  of  the  potential  weaknesses  of  using  listserv  as  a means  of  class  discussion  is  that  it  would  result  in  dry 
dialogue  devoid  of  emotion  that  would  be  found  in  classroom  discussion.  Examination  of  the  communications 
revealed  several  instances  (54)  of  humor  injected  into  the  discussion.  Some  of  these  were  only  emoticons  such  as, 
“<G>”  for  “grin”  or  “:-)”  for  “smile”  but  other  messages  were  intended  to  be  humorous,  and  they  were.  For 
example,  one  student  asks,  “The  peanut  gallery  would  like  to  propose  a question  to  the  powers  that  be.”  Dr. 
Morrison  later  responds,  “Deborah,  is  she  talking  about  us????”  Another  instance  of  humor  is  found  when  a student 
states,  “Unfortunately  it  would  be  easier  to  break  into  Fort  Knox  than  a Canadian  school  in  July.”  Further  humor  is 
demonstrated  in  this  response,  “Jeff  squirms  in  mock  pain:  Excellent  retort.  I am  proud  of  my  personality  disorder. 
It  has  made  me  kind  of  a legend  at  my  school.  Weren’t  all  those  HyperCard  stacks  a form  of  programmed 
instruction?  Hmm?”  One  last  example  of  humor  is  shown  in  this  response,  “I  am  not  a behaviorist  nor  a cognitivist. 

I am  a FANIST.  Consider  the  needs  of  your  students  and  use  the  most  appropriate  methods,  and  it  will  always  PAN 
out.” 

Another  example  of  expressing  emotion  was  this,  “Behaviorist  and  Constructivist  differ  on  picture 
perception  (insert  sarcasm  here.)” 

There  was  evidence  that  students  changed  their  point  of  view  because  of  the  listserv.  “Now  that  I have  a 
connection,  let  me  answer  once  and  forever  about  what  has  dominated  the  previous  discussions.  I do  have  strong 
feelings  about  the  comparisons,  but  have  changed  my  opinion  considerably ” 

There  were  also  some  hurt  feelings  (2)  reversed  in  the  communications.  Well  into  the  seminar  one  student 
exclaims,  “Please  start  spelling  my  name  right!”  Oi  nother  occasion  a student  responds,  “You  wound  me  deeply.  I 
have  a great  deal  of  respect  for  Dr.  X.” 

Analysis  of  Questionnaire 

The  questionnaire  was  answered  by  six  of  the  eight  students  in  the  class.  Attempts  were  made  to  complete 
the  questionnaire  by  telephone  for  the  two  non-respondents,  but  this  attempt  was  not  successful.  Table  1 
summarizes  the  responses  to  the  questionnaire. 

The  students  were  unanimous  in  liking  the  listserv  discussion  because  it  gave  them  more  time  to  think 
about  their  contribution.  One  student  stated,  “I  liked  being  able  to  think  over  my  answers  before  giving  them.  I felt 
that  my  level  of  interaction  was  deeper  because  I wasn’t  giving  instant  answers  to  questions  posed.”  Another 
student  liked  using  the  listserv  because  she  felt  more  comfortable  with  it  than  with  face-to-face  discussion. 

Reasons  for  disliking  the  listserv  discussion  were  that  it  was  very  time  consuming,  it  was  sometimes 
difficult  to  follow  the  discussion,  and  there  was  a lack  of  visual  and  auditory  nuances  found  in  a classroom 
discussion.  One  student  stated,  “I  found  that  sometimes  I unintentionally  irritated  a few  people  simply  because  they 
didn’t  have  enough  cues,  visual  etc.  or  auditory,  tone  of  voice  to  perceive  my  statements  in  the  manner  in  which  they 
were  intended.” 

The  most  valuable  aspect  of  the  listserv  discussions  mentioned  hy  four  of  the  six  respondents  was  its 
convenience.  They  were  not  bound  to  appear  in  a class  at  a certain  time,  thus  allowing  them  to  pursue  other 
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activities  and  participating  in  the  class  discussion  to  fit  their  own  schedule.  It  also  eliminated  geographical  problems 
and  allowed  one  student  to  participate  from  Canada.  Another  student  felt  the  listserv  was  valuable  because  it  forced 
constant  preparation  in  order  to  participate  in  the  discussion.  One  practical  aspect  of  using  a listserv  mentioned  was 
that  it  allowed  an  accurate  hard  copy  transcript  of  the  class  discussion. 

Some  of  the  drawbacks  to  using  listserv  that  were  mentioned  were  an  unfamiliarity  with  e-mail  by  one 
student,  and  difficulty  in  getting  access  to  a computer  terminal  by  another  student.  Two  students  found  no 
drawbacks. 

Improvements  suggested  for  using  the  listserv  included  keeping  the  discussion  on  the  topic,  limiting  the 
time  for  discussion,  being  sure  everyone  participated,  and  not  being  required  to  lead  the  discussion  for  a full  week. 

The  students  generally  felt  that  the  role  of  the  faculty  should  be  that  of  a facilitator  in  the  discussion.  Two 
students  would  have  liked  the  faculty  to  participate  more  than  they  did.  The  two  faculty  members  contributed  130 
messages  out  of  the  total  of  594  for  the  course.  Two  students  mentioned  that  they  thought  it  important  for  the 
faculty  to  keep  the  discussion  focused  on  the  topic. 

When  asked  if  they  would  enroll  again  in  a class  conducted  using  a listserv  for  discussion,  four  students 
responded  positively,  one  was  uncertain,  and  another  stated,  yes,  if  it  was  the  only  option.  The  majority  of  the 
students  believed  that  a face-to-face  class  meeting  was  essential,  that  the  class  should  be  restricted  to  about  10 
students,  and  that  it  probably  would  not  work  well  for  undergraduates.  If  the  class  were  international  in  scope,  it 
would  be  important  to  have  background  information  on  the  other  students. 


Table  l.  Summary  of  Student  Questionnaire 
(Not  everyone  responded  to  every  question?) 


Question 

Response 

Frequency 

Example 

1 . What  did  you  like 

(a)  gave  time  to 

6 

(a)“I  liked  being  able  to  think  over  my  answers 

about  using  e-mail 
instead  of  personal 
class  interaction? 

reflect  on  answers 

before  giving  them.  I felt  my  level  of 
interaction  was  deeper  because  I wasn't  giving 
instant  answers  to  questions  poised.” 

(b)  relieved 
uneasiness  about 
class  discussion 

1 

(b)  ”1  do  not  really  enjoy  discussion  classes 
because  I am  not  comfortable  in  them  and  e- 
mailing  took  away  some  of  the  uneasiness.” 

2.  What  did  you 

(a)  discussion  hard  to 

2 

(a)  “The  main  component  I disliked  about  c- 

dislike  about  using  c- 
mailing  instead  of 
personal  class 
interaction? 

follow 

mail  was  the  lack  of  case  in  following  the 

discussion it  was  hard... .to  remember  or  to 

tell  who  was  sending  the  e-mail.” 

(b)  lack  of  visual  and 
auditory  cues 

n 

(b)  “I  found  that  sometimes  I unintentionally 
irritated  a few  people  simply  because  they 
didn’t  have  enough  cues,  visual  etc.  or 
auditory,  tone  of  voice  to  perceive  my 
statements  in  the  manner  in  which  they  were 
intended.” 

(c)  took  more  time 

2 

(c)  “It  seemed  as  if  it  took  more  time  to  read 
all  the  messages  and  I got  overwhelmed.  Also 
it  limited  my  involvement” 

(d)  personal  behavior 

1 

(d)  “...another  member  of  the  class  and  I had  a 
tendency  to  ‘gang  up’  on  certain  class 
members.” 
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3,  Describe  the  most 
valuable  aspects  of 
being  involved  in  the 
listserv  discussion. 


4.  Describe  the  least 
valuable  aspects  of 
being  involved  in  the 
listserv  discussion. 


(a)  convenience  of  4 
not  having  class 
meetings 

1 

(b)  forced  preparation 


(c)  doing  something 

new ' * 

1 

(d)  have  accurate 

transcript 

(a)  not  accustomed  to  1 
e-mail 


(b)  access  to  a 2 

terminal 


(a)  “...the  convenience  of  not  having  to  come 
to  scheduled  class  meetings.” 

(b)  ‘The  self-imposed  pressure  to  be 
‘prepared’  at  all  times  through  keeping  up  and 
reading  more  than  what  was  required.” 

(c)  ‘‘To  be  in  on  breaking  new  ground-doing 

something  new ” 

(d)  “The  fact  that  I have  100%  accurate 

transcript  of  every  discussion  that  took  place  in 
the  class.” 

(a)  “I  was  not  accustomed  to  sing  e-mail,  so 
the  information  given  at  each  message  was  a 
distraction.” 

(b)  “ I had  to  beg  or  borrow  terminal  time.” 


(c)  none  2 


(c)  “I  don’t  feel  there  is  any  least  valuable 
aspect.” 


(d)  took  more  time 


(d)  “The  time  involved  each  day  to  read  the 
message.  It  curbed  my  interaction.” 


(e)  lack  of  personal 
interaction 


(e)  “Lack  of  personal  (face-to-face) 
interaction.” 
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5.  How  could  the  e- 

(a)  make  sure 

1 

(a)  “Make  sure  that  everyone  on  the  listserv 

mail  discussion  be 

everyone 

participates.. 

improved? 

participates 

(a)  “It  was  irritating  that  some  classmates  could 
Murk’  with  only  occasional  comments  and  get 
the  same  grade  as  those  of  us  who  put  a lot  of 
effort  into  the  class.” 

(b)  not  lead 

1 

(b)  “The  responsibility  for  ‘carrying’  the 

discussion  for  full 

discussion  for  a solid  week  was  overly  time 

week 

consuming  and  could  be  less.” 

(c)  time  limit  on 

2 

(c)  “Put  a cap  on  the  types  of  replies.  A lot  of  it 

discussion 

was  meaningless  conversation  at  times.” 

(d)  need 

1 

(d)  “...setting  up  guidelines  on  how  to  respond  to 

guidelines  on  how 

each  other  such  as  copying  in  the  original 

to  respond 

comments,  etc.” 

1 

(e)  “...emphasizing  the  importance  of 

(e)  introduce 

introducing  the  subject  and  respondent  at  the 

subject  & 

beginning  of  each  message.” 

respondent 

3 

(0  “I  think  the  professors  involved  let  the 

(0  discussion  got 

discussion  get  off  topic  and  somewhat  out  of 

off  topic 

control  from  time  to  time.” 

6.  What  role  do  you 

(a)  assume  role  of 

3 

(a)  “The  faculty  should  play  the  role  of 

think  the  faculty 
should  play  in  the 

facilitator 

facilitator.” 

course  discussion? 

(b)participate 

2 

(b)  “I  think  faculty  could  participate  in  the 
discussion  more  through  posing  questions, 

more 

scenarios,  etc.” 

3 

(c)  “To  keep  the  listserv  on  topic.” 

(c)  keep  on  topic 

7.  How  would  you 
evaluate  the  faculty 

(a)  adequate 

3 

(a)  “Faculty  participation  v/as  adequate.” 

participation  in  the 
listserv  discussion? 

(b)  less 

1 

(b)  “Less  than  I would  like.” 

(c)  excellent 

1 

(c)  “Excellent!” 

8.  Would  you  enroll 

(a)  uncertain 

1 

(a)  “I’m  not  sure  if  I would  enroll  in  another  e- 

for  another  class 

mail  course.  It  would  depend  on  the  class  and 

using  this  method  of 

the  faculty  involved.” 

class  interaction? 

(b)  definitely  yes 

4 

(b)  “Oh  definitely...” 

(c)  if  only  option 

1 

(c)  “If  it  were  the  only  option.” 
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9.  Do  you  think  some 
class  meetings  arc 
essential  or  could  the 
class  interaction  be 
entirely  by  e-mail 
with  no  meeting? 

(a)  yes 

(b)  no 

5 

1 

(a)  “I  felt  more  comfortable  knowing  who  I am 
responding  to  and  being  able  to  connect  a face  to 
the  c-mail  message?’ 

(b)  “No  meetings  are  necessary.  Wc  got  the 
same  results  whether  we  met  or  not.” 

10.  How  large  of  a 
class  do  you  believe 

(a)  6-10 

4 

(a)  “At  most  10.” 

this  mode  of 
communication 
would  accommodate? 

(b)  10-20 

(c)  larger  is  better 

1 

1 

(b)  “Ideally,  10-20  because  of  the  number  of  e- 
mail  messages  that  would  have  to  read.” 

(c)  “In  fact,  the  larger  the  group  the  richer  the 
discussion.” 

1 1.  Do  you  think  this 
means  of 

(a)  probably  no 

5 

(a)  “No,  classes  are  too  large.” 

communication  could 
be  done  at  an 
undergraduate  level? 
Please  explain  your 
answer. 

(b)  yes 

1 

(b)  “Yes,  if  the  motivation  can  be  instilled.” 

12.  If  this  method  of 
communication  were 
done  on  an 
international  and 
intercultural  level,  do 

(a)  need  to  know 
backgrounds 

4 

(a)  “I  think  background  information  would  be  a 
necessity  because  you  would  most  certainly 
encounter  communication  problems  related  to 
cultural  assumptions  and  ignorance.” 

you  think  that  a 
certain  amount  of 
background 
information  about  the 
participants  be 

(b)  agree  on  style 
of  writing 

1 

(b)  “ A conventional  style  of  writing  should  be 
agreed  upon  and  there  might  need  to  be 
additional  discussions  concerning  the  nature  and 
involvement  of  the  course.” 

necessary  or  desirable 
to  help  in 
communication? 

What  type  of 
information  would  c 

(c)  need  to  be 

personally 

acquainted 

1 

(c)  “ ...I  think  that  the  members  of  the  class  all 
need  to  be  personally  acquainted  for  this  to 
work.” 

beneficial? 

(d)  possible 
language  barrier 

1 

(d)  “Yes,  the  language  would  be  a barrier.” 
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13.  Do  you  have  any  (a)  none  2 

additional  comments? 

(b)  enjoyed  i 

(b)  “I  thoroughly  enjoyed  the  listserv  class  last 
summer...” 

(c)  good  2 

(c)  “It  was  a wonderful  experience  that  I highly 

experience 

recommend  to  anyone  looking  for  a class  which 
is  a little  different.” 

(d)  need  only  1 1 

(d)  “I  think  only  one  faculty  member  is  needed 

teacher 

for  such  a course.” 

(e)  have  practice  1 

(c)  “Have  a short  practice  session  before  the  first 

session 

round  of  discussion.” 

(f)  have  minimum  1 

(0  “Have  a minimum  and  maximum  number  of 

& maximum 
number  of  replies 

replies  for  each  participant.” 

Summary  of  Faculty  Interviews 

Both  of  the  faculty  members  believed  this  course  was  successful  in  achieving  its  purpose.  Dr.  Morrison 
slated,  “So,  overall  I think  we  introduced  them  to  some  new  ideas,  new  concepts,  so  they  changed  their... own  belief 
systems,  which  is  good.  So,  overall  I think  it  was  a success.”  Dr.  Lowlher  also  believed  the  course  was  successful, 
and  to  support  this  she  stated,  “From  their  responses  we  could  sec  that  they  were  learning....  We  thought  it  was  very 
successful.” 

The  faculty  were  also  in  agreement  that  the  great  advantage  of  conducting  the  class  discussion  by  listscrv 
was  its  convenience.  “It  gives  you  the  advantage  in  that  you  can  participate  at  any  lime  during  the  day  and  during 
the  week,”  Dr.  Morrison  observed,  and  he  also  pointed  out  , “Taking  the  PowerBook  with  us,  we  could  call  (from) 
any  place  in  the  country  and  still  participate  and  manage  the  instruction.” 

Another  advantage  mentioned  by  Dr.  Morrison  was  that  it  gave  the  students  lime  to  form  thoughtful  replies. 
“People  had  a chance  to  let  ideas  gel,  to  read  the  stuff  and  go  back  and  read  it  (again).  In  a conversation  it  comes  by 
you  one  time,  and  that’s  it.”  Dr.  Lowthcr  observed,  the  students  “liked  the  idea  of  being  able  to  compose  their 
thoughts.” 

Dr.  Morrison  pointed  out  an  advantage  not  alluded  to  by  anyone  else,  in  that  the  discussion  could  be 
personalized.  “You  can  also  carry  on  a one-to-one  conversation.  So  if  someone  disagrees  with  you,  or  they  don’t 
understand,  then  you  have  more  time  to  tutor  them  or  to  explain  your  ideas,  your  thoughts.” 

The  only  disadvantage  the  two  faculty  members  found  was  that  the  listscrv  discussion  was  very  demanding. 
“It  was  24  hours  a day,  7 days  a week.  I mean  e-mail  was  going  continuously,  and  if  you  skipped  a day,  you  were 
out  of  luck.”  Dr.  Lowther  said,  “It  was  very,  very  lime  consuming  to  read  all  of  the  messages  that  the  students  were 
sending.” 

One  suggestion  to  improve  future  classes  using  listscrv  for  discussion  was  ;o  limit  and  schedule  the  lime  for 
discussion.  Both  faculty  members  concurred  in  this.  The  faculty  members  were  also  in  agrecmcnl  concerning  their 
advice  for  future  classes  using  listserv.  They  would  limit  the  enrollment  to  a size  that  is  manageable  for  everyone. 
The  class  should  be  one  that  is  content  rich  and  where  discussion  is  important  and  not  dependent  on  lecture  for 
presentation.  The  students  should  be  adept  at  using  e-mail,  and  it  :s  helpful  if  they  arc  acquainted  with  each  other. 
If  they  arc  not  acquainted,  some  biographical  information  would  be  hUpful. 
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Table  2,  Summary  of  Faculty  Interviews 


Question 

Answer 

Quote 

What  made  you  decide  to  conduct 
the  class  discussion  by  Listserv? 

Convenience  and  flexible  hours 

“ It  sort  of  opened  the  classroom  up 
and  gave  us  the  chance  to  hold  the 
class  without  disrupting  anybody’s 
life  schedules  for  the  summer.” 

Dr.  Morrison 

“It  was  convenience.”  Dr.  Lowthcr 

What  were  the  goals  you  had  for  this 
course? 

Introduce  the  students  to  classical 
instructional  design  literature 

“We  wanted  to  introduce  them  to 
some  literature  that  we  may  have 
just  touched  on  in  the  classes  but  had 
not  gone  into  in  detail.  This  gave 
them  a chance  to  read  articles  and 
talk  and  discuss  the  article.”  Dr. 
Morrison 

“We  thought  that  to  get  that 
knowledge  and  understanding  was  to 
do  quite  a bit  of  reading  and  then  to 
do  some  analysis  after  the  reading 
....then  to  discuss  with  your  peers 
some  of  the  ideas  you  had.” 

Dr.  Lowthcr 

How  would  you  describe  your 
success  in  reaching  these  goals? 

They  learned 

“They  were  able  to  carry  on  an 
intelligent. ..what  I thought  was  an 
intelligent  conversation.  They 
changed  their  opinions  about  some 
of  the  concepts  and  ideas  in  those 
articles.”  Dr.  Morrison 

“From  their  responses  wc  could  sec 
that  they  were  learning.”  Dr. 
Lowthcr 
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Describe  the  advantages  of 
conducting  the  class  discussion  by 
listscrv  instead  of  in  person? 


Strengths: 
Flexible  hours 


Flexible  location 


Time  for  thought 


Discussion  could  be  personalized 


Convenience 


More  accountable 


“It  gives  you  the  advantage  in  that 
you  can  participate  at  any  time 
during  the  day  and  during  the  week.” 
Dr.  Morrison 

“Taking  the  PowerBook  with  us,  we 
could  call  any  place  in  the  country 
and  still  participate  and  manage  the 
instruction.”  Dr.  Morrison 

“But  people  had  a chance  to  let  the 
ideas  gel,  to  read  the  stuff  and  go 
back  and  read  it,  where  in  a 
conversation  it  comes  by  you  one 
time  and  that’s  it.”  Dr.  Morrison 

“They  (students)  liked  the  idea  of 
being  able  to  compose  their 
thoughts.”  Dr.  Lowther 

“You  can  also  carry  on  a one  to  one 
conversation.  So,  if  someone 
disagrees  with  you  or  they  don’t 
understand,  then  you  have  more  time 
to  tutor  them  or  to  explain  your 
ideas.. .your  thoughts.”  Dr.  Morrison 

“It  was  convenience.”  “Students 
really  liked  being  able  to  stay 
home.”  Dr.  Lowther 

“ it  was  in  written  form  made 

them  more  accountable  for  what 
they  say.”  Dr.  Lowther 
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“Instead  it  was  24  hours  a day  7 
days  a week.  I mean  e-mail  was 
going  continuously.  And  if  you 
skipped  a day  you  were  out  of  luck/’ 
Dr.  Morrison 

“It  was  very,  very  lime  consuming  to 
read  all  of  the  messages  that  the 
students  were  sending.”  Dr.  Lowlher 

Misinterpret  e-mail  “...sometimes  you  misinterpret  the  e- 

mail.”  Dr.  Lowther 


What  problems  did  you  experience 

Referring  to  past  statements 

“ The  greatest  problem  was  knowing 

using  this  means  of  communication? 

how  to  refer  to  a past  statement.” 
Dr.  Lowlher 

Learning  lo  summarize 

“Learning  how  to  summarize  what 
someone  said  without  loosing  the 
richness  of  it  without  having  to  paste 
in  the  whole  statement.” 

Dr.  Lowther 

“I  think  that  in  the  future  we  need  to 
look  at  ways  of  limiting  that. 
Like. ..We’ve  talked  about  Monday, 
Wednesday,  Friday,  or  Tuesday, 
Thursday,  Sunday  are  the  only  days 
you  can  post  responses” 

Dr.  Morrison 

Guidelines  “There  needs  to  he  some  guidelines 

on  the  length  of  what  you  can  do.” 

Dr.  Lowther 


How,  as  an  instructor,  did  you  Twenty  percent  of  grade  was  “We  sort  of  kept  an  anecdotal 
evaluate  each  participant’s  discussion  reports  or  journal  during  the  week  as 

contribution?  to  who  was  communicating,  what 

was  going  on  and  who  was  changing 
ideas.” 

Dr.  Morrison 

“ Wc  would  just  watch.  And  see 
what  was  going  on.”  Dr.  Lowther 

“We  had  an  overall  assessment  lo 
where  they  turned  in  papers." 

Dr.  Lowther 
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How  could  the  e-mail  discussions  Suggested  time  frame 
have  been  improved? 


Describe  the  disadvantages  of  Weaknesses: 
conducting  the  class  discussion  by  Demanding  in  time 
listserv  instead  of  in  person? 
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Do  you  think  the  listserv  is  suitable 
for  any  class  or  is  it  limited  to 
certain  types  of  instruction?  If  so, 
what  are  the  limitations? 

Adapting  the  listserv  to: 
Discussion  class: 

“Discussion  type  class  seems  best 
suited  for  Listserv.”  Dr.  Morrison 

“I  think  it  has  to  be  a class  that  is 
content  rich.”  Dr.  Lowther 

Lecture  class: 

“In  say  a typical  lecture.. .standing 
up  lecturing  and  students  listening, 
taking  notes,  I think  you  could  have 
the  listserv  going  as  a discussion  of 
the  lectures.  Particularly  if  the 
lecturer  would  post  his  or  her  notes 
or  some  questions.  Then  you  could 
have  the  students  discuss  those 
questions  related  to  the  lecture.” 

Would  you  consider  conducting  this 
class  entirely  by  Listserv? 

Wc  did  that 

“Well,  that  is  sort  of  what  we  did 
with  this  cne.  Since  it  was 
discussion  that  worked  out  just 
fine.”  Dr.  Morrison 

“Yeah,  we  did  it  with  this  one.” 
Dr.  Lowther 

How  large  of  a class  would  this 
listserv  be  suited  for? 

There  is  an  upper  limit 

“I  think  there’s  probably  an  upper 
limit.  If  you  got  above  20  it  may  just 

get  too  humongous.” 
Dr.  Morrison 


“I  think  you  could  possibly  do  it 
with  15.”  Dr.  Lowther 

“If  you  had  a class  of  300  you  might 
want  to  have  many  separate 
individual  listserves.”  Dr.  Morrison 

“...with  300  people  you  would  have 
too  many  lurkers.  There  is  no  way 
they  could  all  participate.  ...but  you 
could  have  specific  groups  that  had 
to  actively  participate  during 
specific  times  in  the  semester  while 
the  rest  are  lurkers.”  Dr.  Lowther 
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How  much  participation  by  the 

teacher  should  occur?  Participant 


“You  can  be  a participant  and  join  in 
the  discussion  and  present  your 
points  of  views,  and  questions  other 
people’s  point  of  views.” 

Dr.  Morrison 

Facilitator  4,Or  you  can  basically  settle  back 

and  do  nothing  until  someone  gets 
out  of  hand.”  Dr.  Morrison 

“I  saw  my  role  as  a guide. ..a  mentor. 
A facilitator  not  an  instructor.” 

Dr.  Lowther 

What  is  the  role  of  the  teacher  in  Multi-faceted  “It’s  multi-faceted.”  Dr.  Morrison 

leading  the  discussion?  

All  the  participants  were  computer  Competent  with  e-mail  “Well,  they  have  to  be  pretty 

competent.  What  prerequisite  should  competent  with  e-mail.” 

a student  have  for  this  course?  to  Dr.  Morrison 

what  extent  must  the  student  be 

computer  competent?  “They  need  to  know  how  to  use  e- 

mail.  Save  a document,  create  it, 
send  it,  and  be  able  to  open  it  and 

reply.”  Dr.  Lowther 

If  you  were  to  conduct  this  class  List  the  times  “I  think  the  big  thing  that  we  will  do 

again  and  use  the  listscrv  for  the  this  summer  is  to  list  the  times  that 

discussion,  what  would  you  do  you  can  post.”  Dr.  Morrison 

differently  to  improve  the 

experience  for  the  student?  “I  think  a little  more  structure a 

little  instruction  about  e-mail.  Some 
e-mail  etiquetie  maybe.”  Dr. 

Lowther 

Summary  and  conclusions 

The  listscrv  communications  that  occurred  during  the  course  of  this  seminar  clearly  demonstrated  that  this 
method  of  discussion  was  successful  for  the  purpose  of  this  class.  Thematic  analysis  of  the  posts  and  appli'  ..lion  of 
Mason’s  (1991)  criteria  for  evidence  of  critical  thinking  in  computer  communications  gave  evidence  that  the 
students  were  critically  analyzing  and  synthesizing  the  course  reading  material. 

Besides  critical  thinking,  the  messages  also  revealed  the  emotions  of  humor,  hurt  feelings  and  sarcasm 
although  the  participants  felt  that  this  was  not  comparable  to  the  subtle  auditory  and  visual  nuances  present  in  facc- 
to-lacc  discussion.  Misunderstandings,  which  were  infrequent,  were  due  to  content  of  messages  and  not  due  to  the 
electronic  media  being  used. 

This  method  of  class  discussion  had  an  advantage  over  face-to-face  discussion  in  that  it  allowed  the 
participants  greater  opportunity  to  consider  their  responses  before  contributing  their  reply.  It  also  had  the  advantage 
of  convenience  because  the  participants  did  not  have  to  be  in  a class  at  a certain  time  or  in  a certain  place. 

The  main  disadvantage  in  using  listscrv  for  discussion  was  that  it  was  very  demanding  and  time  consuming. 
Another  drawback  mentioned  by  several  students  was  that  they  missed  the  visual  and  auditory  nuances  present  in  a 
face-to-face  discussion. 

In  future  courses  using  listserv,  scheduling  at  time  for  ihe  discussion  would  help  to  solve  the  problem 
presented  by  a continuous  flow  of  communication.  Familiarity  with  the  use  of  e-mail  and  ready  access  to  a computer 
arc  important  prcrcqulsi'es  for  students  in  the  course.  It  is  also  important  for  the  faculty  to  keep  the  discussion 
focused  on  topic  and  to  insure  everyone  participates  while  not  allowing  a few  students  to  dominate  the  discussion. 


Undoubtedly  the  availability  of  lislscrv  for  use  in  class  discussion  will  make  it  possible  to  extend  the 
availability  of  some  courses  of  study  and  to  enrich  them  as  well  by  providing  greater  access  to  educational 
opportunities.  As  computer  usage  by  students  increases  and  availability  of  avenues  of  communication  broaden*  the 
use  of  this  media  for  educational  purposes  will  increase.  It  is  important  for  educators  to  take  full  advantage  of  the 
potentialities  of  the  advances  in  communication. 

Although  the  use  oi  listserv  was  successful  for  the  purpose  of  this  seminar,  there  arc  still  many  questions 
regarding  the  use  of  listserv  that  would  occur  in  other  classes.  For  example,  what  is  the  optimum  class  size  for  use  of 
this  media?  What  are  the  upper  and  lower  limits  of  enrollment  in  such  a class?  Do  the  instruction  methods  of  the 
faculty  need  to  be  adapted  to  this  change  in  method  of  communication,  and  if  so,  what  are  they?  Arc  some  students 
better  suited  for  listserv  discussion  while  others  face-to-face  discussion?  If  so,  how  do  we  determine  this?  Could 
some  subjects  be  improved  by  listserv  discussion?  For  example,  a class  in  political  science  might  benefit  from 
participation  of  students  in  other  countries.  All  of  these  questions  need  further  research  in  order  to  make  the  best  use 
of  this  method  of  communication. 
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Abstract 

Fourth  grade  elementary  school  children  exhibiting  High  and  Low  mathematics  anxiety  were  trained  on 
multiplication  facts  using  a computer  program  designed  to  bring  their  performance  to  the  automaticity  level. 
Mathematics  anxiety,  measured  by  the.  Mathematics  Anxiety  Rating  Scale  - Elementary  version  (MARS-E),  war 
assessed  before  and  after  the  students  demonstrated  automaticity  level  performance  on  the  multiplication  facts. 
Results  showed  that  all  of  the  students  automatized  the  multiplication  facts  using  computer  training.  Students  in  ;ne 
High  Anxiety  group  averaged  the  greatest  improvement  in  performance,  and  were  indistinguishable  from  the 
performance  of  the  Low  Anxiety  group  by  the  end  of  the  automaticity  training.  The  High  Anxiety  girls,  but  not  the 
High  Anxiety  boys  significantly  reduced  their  mathematics  anxiety  ratings.  No  significant  change  in  mean  anxiety 
ratings  were  detected  for  students  in  either  of  the  Low  Anxiety  groups  or  the  Control  groups.  Results  of  the 
experiment  indicate  that  both  High  and  Low  Anxiety  boys  and  girls  achieved  automaticity  level  performance  of 
multiplication  facts  using  CAI  training,  and  training  of  multiplication  facts  to  the  automaticity  level  resulted  in 
significant  reductions  of  mathematics  anxiety  ratings.  These  results  provide  support  for  the  position  that 
mathematics  anxiety'  may  result  from  a failure  to  learn  or  inadequate  preparation  in  the  mastery  of  fundamental 
skills.  Implications  of  these  findings  for  mathematics  instruction  and  curricula  development,  in  light  of  the  NCTM 
Curriculum  standards  are  discussed. 

Mathematics  anxiety  has  been  shown  to  be  a substantial  problem  for  many  students  in  the  United  States, 
Canada,  Great  Britain,  Belgium,  and  other  parts  of  the  world  (Lewis,  Hitch,  & Walker,  1994;  Nimier,  1993).  Studies 
have  shown  that  a disproportionately  greater  percentage  of  females,  compared  to  males,  exhibit  mathematics  anxiety 
characteristics,  and  that  mathematics  anxiety  may  be  responsible  for  detrimental  and  narrowing  influences  on  the 
vocational  and  career  choices  of  both  men  and  women  (Schneider  & Ncvid,  1993).  Mathematics  anxiety  has  been  a 
significant  problem  for  schools.  While  children  do  not  begin  their  education  with  mathematics  anxiety,  many  of 
them  leave  the  educational  system  with  it  (Reyes,  1984).  Despite  the  variety  of  views  and  models  which  have  been 
proposed  to  explain  mathematics  anxiety  (e.g.  Dreger  & Aiken,  1957;  Dutton,  1954;  Mandlcr  & Sarason,  1952)., 
research  has  been  less  productive  in  identifying  the  causal  factors  which  surround  it  (Hembree,  19S8;  1990). 

Consideration  of  the  relationship  between  instruction  and  mathematics  anxiety  is  especially  timely 
considering  the  recently  proposed  NCTM  Curriculm  Standards  (1989),  which  seek  to  greatly  reduce  the  role  of  drills 
and  rote  mastery  tasks  in  favor  of  making  mathematics  'meaningful’  by  emphasizing  the  conceptual  and  contextual 
aspects  of  mathematics  education.  For  some  authors  (Bloom,  1986;  Carnine,  Jones,  and  Dixon,  1994;  Resnick  & 
Lord,  1981),  students'  development  of  negative  altitudes  and  frustration  toward  mathematics  may  result  from  the 
failure  of  schools  to  promote  the  learning  of  fundamental  mathematics  skills  t*i  mastery  levels.  The  NCTM 
proposal  has  not  been  uniformly  endorsed  as  the  most  suitable  approach  to  mathematics  reform  (Carnine,  Jones,  and 
Dixon,  1994).  Indeed,  the  controversy  surrounding  drill  and  practice  has  waged  unresolved  for  quite  some  time  (e.g. 
Bloom,  1986;  Bryan  & Harter,  1899;  Carnine,  Jones,  and  Dixon,  1994;  Rcsnick  & Ford,  1981;  Reeve,  1944).  Some 
authors  have  suggested  that  the  dilemma  surrounding  student  mathematics  competencies  may  have  been  inherited  as 
a by-product  of  the  math  revolution  of  the  1960s  which  dc-cmphasizcd  drill  and  practice  (Rcsnick  & Ford,  1981), 
while  others  (Carnine,  Jones,  and  Dixon,  1994),  have  argued  that  the  goals  and  objectives  proposed  in  the  NCTM 
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curriculum  standards  (1989)  arc  virtually  the  same  as  those  endorsed  during  the  mathematics  revolution  of  the 
1960's. 

The  importance  of  drill  and  practice  and  its  relationship  to  the  acquisition  of  expert  performance  has  been  a 
topic  of  considerable  interest  (Salisbury,  1990;  Salisbury  & Klein,  1988).  At  the  beginning  of  the  19th  century, 
Bryan  and  Harter  (1899)  reported  that  the  automatization  of  knowledge  brought  about  by  repetitive  practice  is 
instrumental,  not  only  in  the  acquisition  and  refinement  of  new  skills  for  novice  learners,  but  is  the  foundation  for 
the  mastery  level  performance  shown  by  experts.  The  role  of  the  school  system  was  seen  as  providing  the 
instructional  setting  and  opportunities  for  students  to  develop  subject  matter  expertise  through  mastery  of  it's  details. 
Bloom  (1986),  however,  notes  that  children  come  to  school  poorly  prepared  to  meet  the  challenges  of  learning,  and 
that  school  systems,  in  general,  arc  poorly  equipped  to  be  even  less  than  moderately  successful  in  developing 
mastery  level  subject  performance  among  it's  students.  Thus,  given  the  current  de-emphasis  on  drill  and  practice 
activities  in  public  school  mathematics  education,  it  is  reasonable  to  anticipate  that  an  eventual  outgrowth  of  this 
failure  to  provide  adequate  essential  training  in  basic  mathematics  skills  would  be  manifested  in  the  form  of 
mathematics  anxiety.  Mathematics  anxiety,  therefore  is  viewed  as  resulting  from  a basic  lack  of  knowledge,  rather 
than  the  result  of  a pre-existing  anxiety  disposition,  as  has  been  suggested  by  others  (c.g.  Mandler  & Sarason,  1952; 
Sarason,  1980). 

The  use  of  computer  assisted  instruction  (CAI)  offers  an  excellent  vehicle  for  the  assessment  of  the  impact 
of  mathematics  drill  and  practice  activities  in  elementary  school  children  and  it's  relationship  to  mathematics 
anxiety.  CAI  technology  could  hold  a great  deal  of  promise  for  providing  mathematics  instruction  to  students  who 
suffer  from  mathematics  anxiety  (Harris  and  Harris,  1987).  In  addition  to  offering  individualization  and  flexibility, 
CAI  based  instruction  is  impartial,  reinforcing,  appeals  to  student  interest,  and  also  offers  a sense  of  privacy  for 
students  who  are  anxious  or  sclf-co:<scious  about  their  performance. 

A number  of  recent  investigations  have  examined  the  role  of  automaticily  in  learning  by  evaluating  the 
performance  characteristics  of  students  trained  on  a particular  task  or  skill  to  a level  defined  as  'automatic.'  For 
example,  Hasselbring,  Goin,  and  Bransford,  (1987;  1988)  using  computer  based  automaticity  training,  found  that 
learning  disabled  children  were  able  to  master  addition  facts  to  the  same  levels  as  non-learning  disabled  control 
children  who  were  given  regular  classroom  instruction  alone.  Similarly,  students  with  'mild  mental  handicaps’ 
benefited,  significantly  from  automaticity  training  of  addition  and  subtraction  facts  using  a computer  assisted  ’drill- 
and-practice’  inslructional  form  (Lin,  Podcll  and  Rein,  1994;  Podell  and  Lin,  1992).  Other  studies  have  examined 
the  pivotal  role  played  by  automatization  of  performance  in  areas  such  as  spelling  (Ormrod  & Lounge,  1990),  and 
reading  (Nicolson  & Fawcett,  1990). 

This  study  examined  the  relationship  between  4th  grade  students’  mathematics  anxiety  before  and  after 
automatization  of  multiplication  facts,  using  a CAI  drill  and  practice  format.  It  was  expected  that  students 
exhibiting  high  mathematics  anxiety,  but  not  low  anxiety,  would  significantly  reduce  their  mathematics  anxiety 
ratings  following  CAI  training  of  multiplication  facts  to  the  automaticity  level.  Further,  it  was  predicted  that  high 
mathematics  anxiety  students  would  require  significantly  more  trials  to  achieve  the  automatization  of  multiplication 
facts  than  students  exhibiting  low  mathematics  anxiety. 

Subjects 

The  subject  pool  consisted  of  the  total  number  of  4th  grade  students  enrolled  in  a single  public  elementary 
school  located  in  the  Pacific  Northwest.  Fourth  grade  students  were  selected  for  the  experiment  because  pilot 
research  showed  that  these  students  were  still  in  the  initial  stages  of  learning  multiplication  facts,  and  far  from 
achieving  automaticity.  Some  students  did  not  participate  in  the  study  due  to  parental  preference,  and  the  data  for 
other  students  was  not  included  because  the  information  was  incomplete.  A grand  total  of  63  students  (81%) 
completed  the  experiment.  There  were  33  boys  (52%)  and  30  girls  (48%).  The  individual  student  ages  were  evenly 
distributed  across  classrooms.  The  average  age  for  boys  was  10  years  - 1 month,  and  for  girls,  the  average  age  was 
10  years  - 0 months.  Students  receiving  support  through  remedial  services  programs  were  not  included  in  the 
experiment  to  further  control  the  characteristics  of  the  population  sample. 


Instrumentation 

The  primary' instruments  utilized  in  the  investigation  included  the  Math  Builder  Program©  (Wittman, 
1994),  and  the  Mathematics  Anxiety  Rating  Scalc-Elemcntary  School  Level  (MARS-E),  (Suinn,  Taylor,  & Edwards, 
1988).  The  Math  Builder  Program©  is  a CAI  program  developed  by  the  first  author  using  the  HyperCard© 
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authoring  system  and  is  designed  to  provide  drill  and  practice  in  basic  mathematics  facts.  The  program  is 
considered  ‘user-friendly’  because  individual  students  arc  capable  of  operating  the  program  following  only  brief 
initial  instruction  in  its  use,  and  distant  supervision  thereafter.  Detailed  information  concerning  the  Math  Builder 
Program  as  well  as  instructions  for  operation  arc  available  from  the  author  (Wittman,  1996).  The  MARS-E  (Suinn, 
Taylor,  & Edwards,  1988)  is  a 26  item  rating  scale  designed  to  measure  the  mathematics  anxiety  construct  in  4th, 
5th,  and  6th  grade  elementary  school  children.  The  MARS-E  was  normed  on  1,119  elementary  school  children 
randomly  selected  from  6 different  school  districts  across  the  Slate  of  Colorado.  Reliability  estimates  for  the 
MARS-E,  using  the  Cronbach's  alpha  statistic,  was  found  to  be  .88.  Construct  validation  data  indicates  that  the 
MARS-E  correlated  significantly  (-.31)  with  the  Stanford  Achievement  Test,  Mathematics  Test,  with  component 
correlations  for  individual  math  skills  (ie.  applications,  computations,  concepts)  ranging  from  -.26  to  -.29.  The 
MARS-E,  and  the  remainder  of  the  family  of  MARS  tests  (i.e.  MARS,  MARS-A)  arc  considered  the  field  standards 
in  the  assessment  of  mathematics  anxiot,f. 

Procedure 

Two  experimenters  were  available  to  work  with  the  students  throughout  the  spring  term  of  the  school  year 
and  terminating  when  the  individual  students  either  met  the  designated  performance  criterion,  or  the  last  week  of  the 
school  year  arrived.  Individual  students  and  teachers  were  not  informed  about  the  specific  pie,.osc  of  the 
investigation  until  it’s  conclusion.  A group  administration  of  the  MARS-E  was  conducted  with  the  students  during 
regular  classroom  hours.  Following  a brief  introduction  each  student  was  given  a MARS-E  protocol.  The  actual 
administration  of  the  MARS-E  took  approximately  20  minutes  for  each  individual  classroom. 

Following  the  pre-test,  a total  of  twenty-four  students  were  randomly  selected  from  the  upper  (n  = 12)  and 
lower  (n  = 12)  thirds  of  the  mathematics  anxiety  distribution,  forming  2 experimental  groups:  High  Mathematics 
Anxiety,  Low  Mathematics  Anxiety.  Each  of  the  two  groups  was  subdivided  evenly  with  respect  to  gender  (6  male, 
6 female)  as  well  as  mathematics  aptitude  based  on  their  pre-existing  4th  grade  Metropolitan  Achievement  Test 
(MAT)  results.  An  additional  12  students  (6  male,  6 female)  were  randomly  selected  from  the  remaining  pool  of 
subjects  to  serve  as  a Control  group  for  possible  changes  in  MARS-E  ratings  resulting  simply  from  the  passage  of 
time.  Students  in  the  Control  group  did  not  participate  in  the  automaticity-training  portion  of  the  experiment.  Any 
student  whose  overall  MAT  mathematics  aptitude  score  was  found  to  be  in  excess  of  1 .0  standard  deviation  above  or 
below  the  mean  for  their  particular  group  was  dropped  from  the  study  and  a different  subject  was  selected  from  the 
pool  of  remaining  student  candidates. 

Day  1 

On  Day  1 of  CA1  training,  students  were  introduced  to  the  Math  Builder  Program  individually  by  the 
experimenter  to  ensure  a thorough  understanding  of  the  program  and  it's  operation.  The  experimenters  were  blind  to 
each  subject’s  mathematics  anxiety  condition  throughout  the  course  of  CAI  training  and  testing,  as  well  as  during 
the  post-test  administration  of  the  MARS-E.  For  the  purpose  of  the  present  investigation  students  worked 
exclusively  on  the  multiplication  component  of  the  Math  Builder  Program. 

Level-I  Problems 

After  familiarization  had  been  completed,  each  student  was  allowed  the  opportunity  to  complete  3 Lcvcl-1 
Multiplication  Problems  for  practice.  Thereafter,  each  student  was  given  daily  practice  on  Lcvcl-I  problems  until  a 
criterion  score  of  29/30  correct  responses  for  each  problem  had  been  achieved  on  two  consecutive  daily  trials. 
Lcvel-I  problems  consisted  of  30  multiplication  facts  were  selected  from  the  standard  matrix  of  multiplication  facts. 
Selection  was  narrowed  to  include  only  one  fact  associated  with  any  fact-pair  (c.g.,  only  one  form  of  the  problem 
pair  3 x 2,  or  2 x 3,  selected  at  random  was  used),  the  limited  use  of  fact-twins  (c.g.,  5 x 5,  or  6 x 6;  only  2 of  the  10 
a ailablc  twins  were  selected  and  used  in  the  experiment),  and  the  elimination  of  0 and  1 number  facts  from 
consideration  (c.g.,  3 x 0,  or  4 x 1).  Information  concerning  the  specific  multiplication  facts  used  in  the  experiment 
is  detailed  in  Wittman  (1996).  The  same  multiplication  facts  were  used  throughout  the  entire  course  of  the 
experiment,  however,  the  order  of  problem  presentation  was  randomized  prior  to  each  session.  Information 
concerning  the  number  of  correct  answers  was  recorded  from  the  student's  score  card  at  the  end  of  each  session  and 
entered  into  a spreadsheet.  Once  a student  met  the  criterion  scurc  of  29/30  on  the  Lcvcl-I  problems  for  2 
consecutive  sessions,  the  student  was  advanced  to  the  Level  II  problems  at  the  next  scheduled  training  session.  The 
requirement  of  2 consecutive  ncar-pcrfcct  sessions  for  the  Lcvcl-1  problems  was  designed  to  meet  the  criterion 
established  by  Hassclbring,  Goin,  and  Bransford  G988),  that  students  have  an  established  declarative  knowledge 


base  for  the  recall  of  mathematics  facts  from  memory.  Students  who  were  observed  using  an  obvious  procedural 
strategy  during  the  Lcvcl-I  phase  of  the  Math  Builder  Program  were  gently  reminded  to  answer  on  the  basis  of 
recall,  and  to  'guess  if  necessary,'  if  uncertain  about  the  answer  to  a problem.  The  criterion  of  29/30  correct  was 
adopted  for  the  experiment  instead  of  the  more  rigorous  30/30  because  pilot  data  indicated  that  most  of  the  students 
in  the  tryout  made  at  least  one  'careless  error'  during  any  given  individual  session. 

Level-II  Problems 

The  Lcvel-11  component  of  the  Math  Builder  Program  was  designed  to  bring  the  level  of  student 
performance  of  basic  multiplication  facts  to  a level  defined  as  automatic.  According  to  Logan  (1985)  simply 
knowing  the  answer  to  a problem  is  different  than  automatization  of  the  response.  In  the  case  of  automatic ity,  the 
qualifying  criterion  refers  to  a specific  ‘effortless’  property  of  performance.  In  research  practice,  the  parameters 
concerning  what  constitutes  the  automatization  of  mathematics  facts  have  been  variable,  ranging  from  one  or  three 
seconds  per  problem,  (Wicbe,  1988;  Hasselbring,  et  al,  1987;  1988).  Initial  test  trials  using  the  Math  Builder 
Program  was  largely  consistent  with  these  parameters.  Pilot  testing  showed  that  the  3-second  interval  comfortably 
reflected  the  student's  uninterrupted  movement  of  the  mouse  to  the  response  button  immediately  after  the  initial 
presentation  of  ihe  multiplication  fact  on  the  computer.  Any  attempt  by  students  to  problem-sol vc  a multiplication 
fact  using  a strategy  other  than  memory  retrieval  (c.g.  counting,  adding  on)  resulted  in  reaction  times  that  were 
always  4 seconds  or  considerably  longer.  In  practice,  the  students  continually  worked  harder  to  earn  increasingly 
faster  reaction  times,  despite  the  3 second  criterion,  and  the  majority  of  the  students  were  responding  to 
multiplication  facts  within  one  or  two  seconds  per  problem. 

The  dependent  variables  to  assess  the  development  of  automaticity  consisted  of  assessing  mean  changes  in 
student  reaction  times  to  individual  multiplication  facts  across  sessions,  as  well  as  a daily  assessment  of  the  overall 
number  of  responses  which  met  the  criterion  set  for  automaticity  across  p.oblems.  In  order  to  meet  this  latter 
objective,  students'  scores  for  each  daily  session  were  converted  to  automaticity  quotients  (Wittman,  1996),  then 
transformed  to  logarithms.  The  formula  for  the  automaticity  quotient  is  given  by  the  following: 


(a/non-a)  x 100  = aq 


where  a = the  number  of  correct  responses  in  a given  daily  session  which  occurred  in  3 seconds  or  less, 
and  non-a  = the  number  of  responses  which  did  not  meet  the  3 seconds  criterion  or  were  simply  incorrect. 


Thus,  for  a fixed  number  of  30  daily  problems 

for  an  outcome  < 50%  automaticity 
for  an  outcome  = 50%  automaticity 
for  an  outcome  > 50%  automaticity 


n = 30 

(5/25)  x 100  = 20  LogAQ  (20)  = 1.30 

( 15/15)  x 100=  100  LogAQ  (100)=  2.00 
(25/5)  x 100  = 500  LogAQ  (500)  = 2.69 


The  LogAQ  characterizes  the  development  of  automaticity  as  a monotonically  increasing  linear 
relationship  between  the  number  of  successfully  automatized  versus  non-automatized  problems  found  in  a iven 
daily  session.  The  LogAQ  transformation  compacts  extreme  scores  into  a more  uniform  distribution  to  better  meet 
the  requirements  for  statistical  tests  (Howell,  1993). 

For  the  Level-II  sessions,  students  were  seated  individually  before  the  computer  and  performed  under 
conditions  identical  to  those  used  for  the  Level-I  condition.  The  Level-II  program  tegan  with  an  initial  set  of 
instructions  emphasizing  the  importance  of  accuracy  and  speed  of  response.  Students  were  reminded  that  the  goal 
of  Lcvel-II  would  he  to  maintain  accuracy  while  responding  with  increasingly  faster  reaction  times  to  each  problem 
presented.  The  experimenter  uniformly  encouraged  students  to  "beat  your  best  time"  throughout  the  Lcvel-II 
sessions.  Level-II  problems  consisted  of  a randomized  list  of  the  same  multiplication  facts  used  in  the  Level-1 
series.  The  problems,  and  the  corresponding  response  buttons  were  randomized  across  presentations  to  ensure  that 
the  students’  learning  reflected  genuine  recall  of  the  mathematics  facts,  and  not  memorization  of  the  pattern  of 
answers.  As  noted  in  the  description  of  the  Level-II  problems,  students  received  visual  feedback  about  their  reaction 
time  for  each  problem  they  solved  correctly.  Criterion  for  automaticity  at  Level-II,  evidenced  by  student  reaction 
times  of  3 seconds  or  less  to  each  of  the  30  individual  multiplication  problems  for  3 separate  session  presentations, 
was  recorded  and  entered  into  a spreadsheet  at  the  end  of  the  day.  Students  practiced  all  30  of  the  multiplication 
facts  until  the  criterion  for  each  of  the  30  multiplication  facts  had  been  met. 
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MARS-E  Post-Test 

Once  each  of  the  24  students  achieved  criterion  performance  on  the  Level-II  multiplication  facts,  the 
MARS-E  was  readministered.  The  students  who  did  not  participate  in  the  CAI  portion  of  the  experiment  were 
simply  read  ministered  the  MARS-E  in  their  classrooms. 

Results 

The  results  of  the  experiment  provided  confirmatory  evidence  for  the  primary  research  hypotheses.  The 
students  achieved  automaticity  level  performance  as  a result  of  CAI  training,  which,  in  turn,  resulted  in  significant 
reductions  in  mathematics  anxiety  ratings  for  students  in  the  High,  but  not  Low  Mathematics  anxiety  groups. 

Analysis  of  Demographic  information 

Independent  analyses  of  the  subject  population  by  MARS-E  Pre-Test  and  by  (CTBS)  Mathematics 
Achievement  failed  to  identify  any  significant  differences  between  groups  as  a function  of  gender,  classroom 
placement,  or  the  interaction  between  these  variables  (all  p's  > .05).  Detailed  information  concerning  these  results  is 
available  in  Wittman  (1996). 

Analysis  of  pre-treatment  variables 

MARS-E  pre-training  scores.  A exploratory  3x2  ANOVA  with  groups  (High  Anxiety,  Low  Anxiety, 
Controls)  and  gender  (Boys,  Girls)  for  the  MARS-E  pretest  variable  demonstrated  a significant  main  effect  of 
anxiety  condition  (F  (2,  27)  = 44.429,  MSE  = 89.89.  p < .0001 ).  Analysis  of  the  main  effect  of  anxiety  using  the 
Ncwman-Keuls  statistic  demonstrated  significant  differences  between  each  of  the  anxiety  conditions  and  the  control 
groups  (all  p's  < .05),  verifying  that  the  deliberate  selection  procedure  of  the  2 anxiety  groups  and  the.  random 
selection  of  control  subjects  resulted  in  clear  differentiation  on  the  mathematics  anxiety  dimension.  Comparison  of 
these  3 groups  to  the  normative  data  provided  by  Suinn,  et  al.  (1988),  demonstrated  that  the  mean  ratings  for 
students  in  the  High  Anxiety  condition  correspondcc  to  the  normative  group  scoring  at  the  95th  %tilc  (+1.5  SD  ), 
the  students  in  the  Low  Anxiety  condition  were  found  to  be  at  or  below  the  10th  %tilc  of  Suinn's  distribution  (-1.5 
SD  ),  and  the  scores  of  the  students  who  were  selected  al  random  to  occupy  the  Control  condition  clustered  closely 
around  the  50th  %lile.  A similar  3x2  ANOVA  calculated  to  examine  the  relationship  between  mathematics 
achievement  variable  and  the  MARS-E  scores,  demonstrated  a significant  main  effect  of  anxiety  condition  (F  (2, 
27)  = 8.05)  p < .001).  The  main  effect  of  gender  (F  (1,  27)  = .018,  ns  ) and  the  interaction  between  gender  and 
anxiety  condition  (F  (2,  27)  = .064,  ns  ),  however,  did  not  prove  statistically  significant.  A Newman-Kculs  analysis 
of  the  main  effect  of  anxiety  group  demonstrated  that  the  High  Anxiety  student  group  manifested  significantly  lower 
CTBS  total  mathematics  achievement  scores  than  students  in  cither  the  control  group  or  in  the  Low  Anxiety 
condition  (p's  < .05).  Students  in  the  control  group,  did  not  differ  significantly  in  math  achievement,  compared  to 
the  students  in  the  Low  Anxiety  condition. 


Differential  performance  of  the  anxiety  groups 

Analyses  of  student  performance  across  the  course  of  the  experiment  showed  that  students  in  the  High 
Anxiety  condition  improved  their  knowledge  and  proficiency  in  solving  basic  multiplication  facts,  significantly,  as  a 
function  of  drill  and  practice.  While  the  High  Anxiety  students'  performance  was  initially  inferior  compared  to  their 
Low  Anxiety  counterparts,  beginning  with  the  Lcvel-I  problems,  the  differences  were  greatly  reduced  across  the 
Lcvcl-11  sessions,  and  eventually  eliminated.  The  performance  of  the  students  in  the  Low  Anxiety  condition,  while 
superior  to  the  High  Anxiety  students  on  Lcvcl-J  problems  and  across  the  course  of  the  Lcvcl-II  problem  series, 
became  quickly  asymptotic,  after  which  the  High  Anxiety  students  narrowed  and  eliminated  tbc  gap.  These  results 
arc  described  in  detail  in  Wittman  (1996). 

MARS-E  Test  Results 

The  results  of  the  MARS-E  pre  and  post  test  ratings  were  analyzed  using  a 2 x 3 x 2 split  plot  factorial 
ANOVA  with  gender  (boys,  girls),  anxiety  condition  (High  Anxiety,  Low  Anxiety,  Comparison),  test  session  (Pre- 
test, Post-test),  and  the  interaction  between  these  variables  serving  as  factors.  Results  of  this  analysis  revealed  a 
significant  main  effect  of  anxiety  condition  (F  (2,  27)  = 24.41,  MSE  - 245.08,  p < .0001),  and  the  3- way 
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interaction  between  gender,  anxiety  condition,  and  test  session  (F  (2,  27)  = 3.64,  MSE  = 210.25  p < .039).  Neither 
the  main  effect  of  gender,  or  any  of  the  other  interaction  terms  approached  statistical  significance  (F's  < 1 ). 

Breakdown  of  the  results  of  the  main  effect  of  anxiety  condition  using  studentized  range  t-scorc  statistic, 
and  the  MSE  for  subjects  error  term  from  the  main  analysis  showed  that,  overall,  the  students  in  the  High  Anxiety 
group  generated  significantly  higher  overall  MARS-E  ratings  than  the  students  in  the  Low  A.nxicty  group  CM  diff  = 
34.72,  t (1,  27)  = 7.27,  SEM  =4.77,  p < .01),  and  students  in  the  Comparison  Group  (M  diff  = 24.43,  t (I,  27)  = 
5.12,  SEM  = 4.77,  p < .01).  The  Comparison  Group's  performance  was  intermediate  in  relation  to  the  High  and 
Low  Anxiety  groups  and  did  not  differ  significantly  from  the  Low  Anxiety  group,  (M  diff  = 10.29,  t (1,  27)  = 
2.15). 

Results  of  the  triple  interaction  (figure  ))  demonstrated  that  the  relationship  between  anxiety  condition  and 
pre-test  versus  post-test  performance,  primarily  reflected  a significant  decrease  in  pre-post  lest  MARS-E  ratings  for 
the  girls  in  the  High  Anxiety  group  {F  (I,  27)  = 9.14,  p < .01).  None  of  the  other  comparisons  within  gender  and 
anxiety  conditions,  across  the  pre  versus  post  test  intervals  came  close  to  reaching  statistical  significance  (all  p's  > 
.05)}. 

Figure  J.  Graph  of  the  gender  x anxiety  condition  x pre/post-test  MARS-E  ratings:  (H  = High  Anxiety,  L = Low 
Anxiety , C = Comparison  Group). 

□ = MARS-E  Pre-test  □ = MARS-E  Post-test 


Mean 

MARS-E 

Rating 


High 

Low  Comparison 

High 

Low  Comparison 

Anx 

Anx 

Anx 

Anx 

Boys 

Girls 

Comparisons  for  girls  at  the  pre-test  interval  showed  that  High  Anxiety  girls  reported  significantly  greater 
mathematics  anxiety  than  the  girls  in  either  of  the  Low  Anxiety  or  Control  group  conditions  (p’s  < .05).  The  girls 
in  the  Control  group,  did  not  differ  significantly  from  the  girls  in  the  Low  Anxiety  condition.  At  the  post-test 
intervals,  however,  the  High  Anxiety  girls  reduced  their  mathematics  anxiety  ratings  to  a level  which  did  not  differ 
significantly  from  the  control  girls  (p  < .05).  The  ratings  of  the  High  Anxiety  girls  were  significantly  higher  than 
the  Low  Anxiety  girls  (p  < .05). 

Similar  analyses  of  the  triple  interaction  for  boys  revealed  significant  differences  between  the  anxiety  and 
control  conditions  at  the  pre-test  as  well  as  at  the  post-tcsl  intervals.  The  nature  of  these  results  was  explored  using 
the  studentized  range  statistic  and  the  pooled  between  MSE  terms  drawn  from  the  main  analysis.  At  the  prctcsl 
interval  the  results  showed  that  the  boys  in  the  High  Anxiety  group  reported  significantly  greater  mathematics 
anxiety  ratings  than  students  in  cither  the  Low  Anxiety  group  or  the  Control  group,  and  the  mathematics  anxiety 
ratings  for  students  in  Ihc  Control  group  were  intermediate  in  relation  to  the  High  Anxiety  and  the  Low  Anxiety 
groups.  The  same  comparisons  between  anxiety  conditions  for  the  boys  at  the  post-test  intervals  showed  that  the 
boys  in  the  High  Anxiety  group  continued  to  report  significantly  higher  mathematics  anxiety  ratings  than  the  boys  in 
the  Low  Anxiety  and  the  Control  groups  (p’s  < .()]),  however,  the  latter  two  groups  no  longer  differed  significantly 
from  one  and  other. 

Analysis  of  the  triple  interaction  for  boys  and  girls  within  each  anxiety  condition,  across  test  sessions 
confirmed  the  results  of  previous  analyses  with  respect  to  the  performance  of  students  in  Ihc  High  Anxiety 
conditions  by  showing  that  only  Ihe  girls  in  the  High  Anxiety  condition  significantly  lowered  their  MARS-E  ratings 
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between  the  pre-test  and  the  post-test  (p  < .05).  Similar  comparisons  examined  for  the  remaining  groups  were  not 
statistically  significant  (all  p's  > .05). 

Discussion 

The  results  of  the  experiment  provided  confirming  evidence  in  support  of  the  main  research  question;  a 
reduction  of  mathematics  anxiety  was  achieved  among  girls  in  the  high  anxiety  condition  presumably  as  a result  of 
automatization  of  multiplication  facts  using  the  CAl  program.  Specifically,  the  girls  in  the  High  Anxiety  condition 
evidenced  a significant  reduction  in  MARS-E  ratings  at  the  post-test,  compared  to  their  pre-test  ratings.  At  the  post- 
test, the  High  Anxiety  girls’  mathematics  anxiety  ratings  were  indistinguishable  from  a group  of  control  girls  who's 
performance  was  found  to  be  within  the  average  range,  compared  to  Suinn,  et  al's  (1988)  MARS-E  normative 
sample.  Thus,  a group  of  girls  who  initially  reported  clinically  debilitating  levels  of  mathematics  anxiety,  at  the  pre- 
test (ic.  95th  %tilc),  subsequently  changed  their  mathematics  anxiety  ratings  to  the  'average  range*  (50th  %tilc),  as  a 
result  of  the  CAI  intervention  and  automatization  of  multiplication  facts. 

While  these  results  are  encouraging,  it  is  not  entirely  clear  why  the  significant  decrease  in  mathematics 
anxiety  rating  was  confined  purely  to  the  girls  in  the  High  Anxiety  group.  Since  the  pattern  of  performance  of  the 
boys  and  girls  in  the  High  Anxiety  group  was  statistically  identical  across  virtually  all  of  the  experimental 
measures,  it  would  seem  reasonable  to  expect  that  parallel  performance  should  have  been  obtained  with  respect  to 
the  mathematics  anxiety  variable,  as  well.  Indeed,  given  the  prevailing  'masculine  attitude*  toward  male  superiority 
in  mathematics  one  would  expect  that  boys  would  be  more  likely  to  understate,  than  overstate  their  mathematics 
anxiety  ratings,  resulting  in  a built-in  degree  of  statistical  regression  of  scores  across  the  course  of  the  experiment. 
Instead,  however,  the  High  Anxiety  boys  appeared  to  be  quite  willing  to  report  their  anxieties  about  mathematics 
and  maintained  this  position  from  pre-test  to  post-test.  A second  and  more  parsimonious  explanation  for  the 
differential  results  could  reside  in  the  very  small  number  of  boys  making  up  the  High  Anxiety  group  who  actually 
completri  the  experiment.  The  subject  mortality  rate  of  50%  is  substantial.  It  would  be  difficult  to  draw  further 
conclusions  concerning  the  mathematics  anxiety  variable  for  these  particular  students  with  a subject  pool  of  only  3 
boys  making  up  the  High  Anxiety  condition. 

Neither  the  students  in  the  Low  Anxiety  or  the  Control  groups  demonstrated  a significant  reduction  in 
mathematics  anxiety  ratings  between  the  pre  and  post  test  intervals  and  this  finding  is  not  surprising.  Since  the 
average  MARS-E  ratings  of  the  Low  Anxiety  group  clustered  around  the  10th  percentile  of  the  normative 
distribution,  any  substantial  improvements  that  they  might  have  made  in  terms  of  MARS-E  ratings  would  have 
easily  been  masked  by  'floor  effects.'  Since  students  in  the  Control  condition  did  not  experience  the  CAI  portion  of 
the  experiment,  changes  in  mathematics  ratings  would  not  be  expected.  Perhaps  of  even  greater  importance, 
however,  was  the  observation  that  none  of  the  students  showed  increased  mathematics  anxiety  ratings  resulting  from 
the  CAI  drill  and  practice  sessions.  Thus,  the  question  concerning  whether  routine  drill  and  practice  activities  might 
be  somehow  anxiety  inducing,  appears  unlikely,  viewed  from  the  standpoint  of  the  performance  of  the  present  4th 
grade  student  group.  The  results  of  the  experiment  provided  encouraging  evidence  to  support  additional  efforts  in 
the  study  of  the  relationship  between  anxiety  variables  and  automaticity. 

The  results  of  the  experiment  hold  important  implications  for  elementary  school  mathematics  instruction, 
particularly  in  light  of  the  1989  National  Council  of  Teachers  of  Mathematics  (NCTM),  curriculum  standards.  In 
addition  to  the  reduction  of  mathematics  anxiety,  other  results  of  the  experiment  (Wittman,  1996)  showed  that 
student  performance  on  multiplication  facts  was  substantially  improved  using  the  Math  Builder  program.  This 
improvement  was  observed  regardless  of  whether  individual  students  exhibited  High  or  Low  Anxiety  levels.  The 
controversy  surrounding  the  best  approach  to  mathematics  instruction,  (ie.  drills  versus  conceptual)  is  beside  the 
point.  In  order  for  students  to  make  mathematics  a skill  to  be  used  meaningfully  for  problem  solving,  students  need 
to  automatize  the  facts  and  routines  to  the  degree  that  conceptual  processing  of  information  occurs  independently  of 
these  basic  processes.  Students’  whose  mathematical  problem-solving  performance  is  hindered  by  the  continual 
need  to  review  facts  and  procedures  will  learn  to  view  mathematics  as  an  anxiety  provoking  experience.  Providing 
students  with  the  opportunity  for  sufficient  drill  and  practice  will  empower  them  with  the  capacity  to  go  beyond  the 
basics;  a process  of  drill  and  discovery. 

The  results  of  the  experiment  provide  support  for  the  position  that  mathematics  anxiety  does  not  necessarily 
exist  as  a personality  characteristic  predating  and  independent-of  mathematics  performance.  Similar  to  Tobias 
(1985),  our  position  suggests  that  mathematics  anxiety  develops  as  a logical  reaction  to  insufficient  learning  of 
fundamental  skills.  Instructional  programs,  and  especially  CAI  programs,  which  emphasize  training  student 
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performance  on  basic  number  facts  to  the  automatic  level  will  provide  students  with  the  skills  they  need  to  become 
confident  mathematicians. 
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Project  Introduction 

In  early  1995,  the  researchers  began  working  with  a second  grade  teacher  and  a group  of  senior  citizens  on  a 
long-term  e-mail-based  project  designed  to  involve  the  seniors  in  the  education  of  a second-grade  class.  After  an 
extensive  planning  period,  the  project  actually  started  in  August,  1996,  and  the  initial  phase  concluded  in  May, 
1997.  The  purpose  of  this  paper  is  to  provide  a description  of  the  activities  and  results  of  this  year-long 
intergenerational  electronic  communication  project. 

Project  Goals 

The  fundamental  purpose  of  the  project  was  for  two  distinct  generational  groups  in  the  same  geographical 
community  to  get  involved  in  a year-long  dialogue  featuring  discussion  topics  generated  by  the  second  grad'.*,  teacher. 
These  topics  were  planned  to  integrate  with  regular  scheduled  student  activities  in  the  classroom.  A key  expectation 
was  that  as  the  seniors  brought  outside  experiences  and  recollections  into  the  second  grade  classroom,  the  students 
would  come  to  see  the  senior  community  as  part  of  their  own,  but  also  that  the  seniors  would  come  to  understand 
and  support  the  school  community.  Consequently,  it  was  also  a goal  of  the  project  that  face  to  face  meetings 
between  the  two  groups  would  also  occur.  These  face  to  face  opportunities  were  identified  as  not  only  the  natural 
outcome  of  the  process,  but  a necessary  part  of  creating  an  intergenerational  community  building  experience.  One  of 
the  key  insights  of  this  project  was  that  the  Internet  could  serve  to  lower  barriers  between  communities  that  were 
geographically  close,  but  socially  unconnected,  i.e.  generationally  distant,  and  enable  such  meetings  to  happen. 

The  researchers  used  Erikson’ s psychosocial  theory  of  development  as  a theoretical  framework  for 
understanding  the  interactions  between  the  two  groups,  and  found  it  to  be  an  extremely  productive  for  that  purpose. 
We  also  framed  our  investigation  in  terms  of  our  goal  of  creating  an  “intergenerational  community.”  In  this  context, 
“intergenerationar  refers  to  an  attempt  to  bond  members  of  two  different  generations  for  the  purpose  of  achieving  a 
goal  which  reflects  the  commonality  of  both.  “Community”  refers  to  a physical  or  virtual  collection  of  members 
whose  existence  is  predicated  on  common  interests,  beliefs,  or  space.  The  goals  implied  in  the  two  terms  provided 
the  researchers  with  a basis  for  observing  and  describing  how  the  project,  through  the  use  of  the  Internet,  was 
beneficial  in  providing  a common  experience  for  both  groups.  In  addition,  Erikson’s  psychosocial  theory  on 
development  was  used  in  generally  assessing  the  distinctive  differences  and  potential  for  connecting  and 
communicating  ideas. 

Erikson’s  Psychosocial  Theory  of  Development 

Erikson’s  psychosocial  theory  of  development  was  developed  as  a way  of  adding  on  to  Freud’s  theory  of 
psychosexual  development  (Miller,  1993).  Unlike  Freud’s  theory  focusing  on  the  stages  of  child  development. 
Erikson  attempted  to  develop  a theory  which  addressed  a life-long  developmental  process  featuring  key  points  of 
psychological  crises  that  needed  to  be  resolved  a one  matures  through  the  eight  stages.  The  crises  points  were 
labeled  as  follows  and  roughly  correspond  to  stages  of  maturity  (Erikson,  1980). 
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Psychological  Crises  Developmental  Stage 


Stage  1-  Trust  vs.  Mistrust 

Infancy 

Stage  2-  Autonomy  vs.  Shame 

18  months  to  3 years 

Stage  3-  Initiative  vs.  Guilt 

3 years  to  6 years 

Stage  4-  Industry  vs.  Inferiority* 

6 years  to  12  years 

Stage  5-  Idendty  and  Repudiation  vs.  Identity 
Diffusion 

Adolescence 

Stage  6*-  Intimacy  and  Solidarity  vs.  Isolation 

Young  Adulthood 

Stage  7-  Generativity  vs.  Seif-absorption 

Adulthood 

Stage  8-  Integrity  vs.  Despair* 

Aging 

* Stages  representative  of  the  two  groups  involved  in  the  project. 


Erikson’s  Psychosocial  Theory  Characteristics  Of  The  Two  Project  Groups 

Although,  Erikson’s  stages  included  eight  monumental  instances  of  development,  this  study  involved 
groups  who  are  primarily  entering  the  fourth  (2nd  graders)  and  eighth  (seniors)  stages  of  Erikson's  model.  In  stage 
four,  industry'  vs.  inferiority,  children  enter  the  larger  world  of  knowledge  and  work.  Learning  about  others,  things, 
and  skills  help  make  up  their  identity.  Likewise,  it  is  very  important  that  guidance  be  given  to  help  alleviate 
frustration  and  failure.  Seniors  have  entered  stage  eight  which  is  characteristic  of  the  conflict  between  integrity  and 
despair.  They  have  begun  the  process  of  accepting  the  limitations  of  life,  and  have  viewed  their  success  in  terms  of 
sharing  a larger  history  that  includes  previous  generations.  A sense  of  commitment  to  the  future  takes  the  shape  of 
increased  interest  in  doing  positive  things  not  for  one's  own  gratification,  but  for  the  sense  of  leaving  a legacy.  This 
legacy  includes  a sense  of  history  that  is  uniquely  needed  in  the  development  of  the  children  operating  in  the  fourth 
stage.  Conversely,  the  seniors  need  the  reinforcement  from  children  to  help  alleviate  the  feelings  of  despair  which 
are  often  characterized  as  regret  for  what  one  has  not  done  in  one’s  life  and  disgust  with  one's  self  and  fear  of  one's 
death. 


Erikson  identifies  community  as  a central  theme  existing  in  both  the  4*  and  8th  stages  concerning  the  radius 
of  significant  relations  (Miller,  1993).  In  the  industry  vs.  inferiority  stage  children  extend  their  concept  of  belonging 
outside  of  the  basic  family  to  include  membership  in  a community.  This  includes  the  psychosocial  modality  of 
making  things  with  others  together  ana  identifying  with  those  accomplishments.  In  the  8th  stage,  community  has 
been  extended  to  include  a more  general  membership  to  mankind.  This  includes  a special  kinship  with  other  8th  stage 
members  who  can  share  a sense  of  survival  as  well  as  a sense  of  wisdom-gathering  that  must  be  handed  down  to 
future  generations.  The  need  for  this  intergenerational  connection  creates  both  a need  to  connect  as  well  as  increased 
feelings  of  alienation.  This  wisdom  is  perceived  by  the  8th  stage, members  to  be  important  yet  unappreciated  by 
members  of  the  other  stages  of  psychosocial  development. 

Second  Grade  Activities  Concerning  Community  Concepts 

Second  graders  in  school  are  typically  in  the  process  of  learning  about  community  and  identifying 
community  members.  This  includes  examples  of  communicating  with  other  second  graders  internatic  nally  to 
experience  what  other  communities  are  like  throughout  the  world.  In  addition,  the  kids  are  learning  about  what 
makes  up  a member  in  their  own  communities.  An  interesting  method  for  teaching  children  community  involves 
lessons  that  attempt  to  characterize  members  by  describing  what  they  do,  such  as  little  league  coach,  preacher, 
fireman-  One  of  the  apparent  negative  results  in  this  teaching  method  is  children  learning  that  active  careers  as  being 
the  most  important  factor  in  determining  members  of  the  community.  Seniors  have  a tendency  to  be  ignored  due  to 
the  fact  that  they  are  no  longer  active  members  providing  a visible  function  in  the  community.  Attempts  at 
introducing  concepts  of  family  with  community  can  be  adopted  to  foster  a connection  with  grandparent  and  seniors 
in  the  community. 
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Components  of  the  Local  Internet  Community 

The  Blacksburg  Electronic  Village  (BEV)  has  been  constructed  to  provide  the  citizens  of  Blacksburg  with 
tools  to  foster  community  involvement.  It  essentially  provides  internet  access  to  a large  number  of  local 
community  members.  One  of  the  most  innovative  concepts  of  the  Blacksburg  Electronic  Village  is  the  ability  to 
provide  a vehicle  in  which  local  community  members  can  interact,  express  concerns,  provide  information  and 
services  for  each  other  with  the  end  result  being  an  electronic  local  social  interaction.  Rogers  (1983)  identified  this 
very  concept  as  being  an  innovation  that  could  bolster  and  redefine  community  membership  in  the  information  age. 
One  of  the  most  active  user  groups  in  the  BEV  community  is  the  senior  citizen  user  group.  The  purpose  statement 
of  the  senior  citizens  group  in  BEV  is  as  follows: 

BEV-Seniors  have  the  primary  goal  of  enhancing  computer  communications  among  senior 
citizens  by  improving  their  ability  to  use  computers  and  BEV  software.  On  the  BEV-Seniors 
Listserv  and  on  the  BEV  "Seniors  Information  Page”  we  also: 

• Discuss  local  topics  in  which  we  have  shared  interests  from  the  perspective  of  being  responsible, 
concerned  senior  citizens. 

• Inform  and  entertain  within  the  limits  of  shared  experiences,  interests  of  our  age  group,  with 
good  taste  and  a civil,  respectful  attitude  for  all  people. 

(Blacksburg  Electronic  Village  Seniors  Purpose  Statement,  1997). 

It  is  clear  that  tne  group’s  emphasis  on  their  civic  role  fits  with  the  features  of  Erikson’s  eighth  stage.  The 
group’s  activities  include  an  active  e-mail  list,  a compilation  of  web  resources  (local  and  national  covering  such  areas 
as  finance,  health  care,  and  computing),  and  homepages  for  a significant  number  of  the  group.  They  also  meet 
monthly  in  the  community  center. 

Adapting  The  Internet  For  An  Intcrgenerational  Educational  Experience 

Conventionally,  the  Internet  is  thought  ot  as  a tool  for  bridging  geographical  distance.  Many  Internet 
projects,  especially  in  the  K-12  arena,  reflect  that  facility,  connecting  kids  with  communities  and  classes  all  over  the 
world,  and  generally  addressing  global  issues  (Burgstanler,  1996;  Ellsworth,  1997,  Wolcott,  1996).  This  has  partly 
been  a product  of  those  institutions  connected  to  the  Internet,  many  of  which  have  been  government  and  education- 
related.  However,  as  the  number  of  private  citizens  and  other  institutions  connecting  via  the  Internet  increases,  new 
possibilities  have  emerged.  One  such  possibility  is  to  bridge  generational  or  sociological  distance,  by  bringing 
together  those  groups  who  are  marginalized  in  the  conventional  structures  of  community  life.  This  has  become 
possible  in  a community  like  Blacksburg,  because  it  is  an  experimental  site  for  the  Electronic  Village  concept.  In 
BEV  all  citizens  have  been  encouraged  and  supported  in  connecting  to  the  Internet,  and  consequently  some  groups 
have  become  very  active.  One  of  these  groups  is  the  senior  citizens,  particularly  as  the  community  is  also 
promoting  itself  as  a retirement  community  (Blacksburg  Electronic  Village  Vision  Statement,  1994). 

Project  Description 

The  project  involved  many  players,  however,  one  of  the  goals  in  implementing  the  project  was  trying  to 
make  the  communication  technology  as  transparent  as  possible.  This  is  especially  important  when  introducing 
technologies  to  younger  students.  In  addition  eachers  are  less  likely  to  take  time  away  from  regular  class  activities 
when  technology  usage  becomes  too  burdensome.  The  project  was  purposefully  designed  with  the  idea  that  the  most 
basic  of  computer  communication  technology  be  used.  Thus  the  primal  source  of  communication  was  the  use  of 
the  classroom  e-mail  system  (second  graders)  and  the  seniors  project  listserv  (senior  citizens),  A listserv  was  set  up 
for  the  interactions  of  this  group.  Students  sent  messages  to  this  listserv  and  seniors  on  the  listserv  responded  as 
they  saw  fit. 

Time  Profile 

The  project  was  designed  to  be  carried  out  throughout  the  school  year.  This  included  both  the  Fall  and 
Spring  Semesters  of  1996  and  1997.  After  assessing  if  there  was  enough  meaningful  exchange,  the  project  would  be 
continued  for  the  following  year  depending  on  willing  participants. 
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Interaction  Protocol 


The  second  graders  worked  in  small  groups  composing  questions  and  comments.  The  seniors  involved 
responded  individually  either  to  a particular  group  of  students,  to  the  list  and  to  any  message  they  wished.  All 
messages  between  student  groups  and  seniors  were  included  on  the  Youth-Senior  Listserv.  Topics  were  selected  by 
the  teacher  and  centered  around  particular  themes  covered  during  the  school  year. 

Supervision  Of  Project 

Although  the  listserv  was  set  up  by  BEV,  it  was  maintained  and  monitored  by  the  researchers.  All 
particular  messages  to  the  students  were  first  read  by  the  teacher.  Any  needless  messages  that  did  not  appear 
appropriate  for  the  students  could  be  deleted  by  the  teacher.  The  teacher  also  supervised  the  messages  composed  by 
the  students. 

Senior  Participation  Of  The  Project 

The  details  of  how  the  project  was  to  be  conducted  and  how  the  seniors  were  to  participate  were  established 
after  lengthy  negotiations  with  the  seniors  and  the  teacher  involved.  The  seniors  group  is  generally  in  high  demand 
for  internet-related  projects  and  they  generally  reject  most  requests.  Typically,  this  is  because  they  do  not  fit  with 
the  seniors  purpose  and  local  orientation.  The  seniors  were  also  concerned  about  their  potential  workload,  since  they 
typically  lead  busy  lives.  They  were  also  concerned  that  the  teacher  take  some  responsibility  to  filter  the  students’ 
messages.  Having  addressed  these  concerns,  we  found  that  the  teacher  was  concerned  that  the  seniors  might  find  the 
experience  too  boring.  Generally  speaking  none  of  these  concerns  emerged  as  a problem.  Once  both  groups 
approved,  the  seniors  were  recruited  from  the  existing  BEV  Seniors  group.  Presentations  about  the  project  were 
made  via  e-mail  and  at  the  group’s  monthly  meeting,  and  about  twenty  seniors  signed  up  for  the  project 

Role  Of  The  Teacher  In  The  Project 

The  project  was  designed  to  enable  the  teacher  to  build  in  the  electronic  intergencrational  communication 
into  the  existing  curriculum.  There,  were  key  themes  that  were  developed  and  served  as  starting  points  for 
communication  activity  between  the  second  graders  and  the  seniors.  This  served  to  further  represent  the  critical 
relationship  between  the  seniors  historical  reflections  and  the  existing  topics  covered  in  the  classroom.  It  was 
considered  important  by  the  researchers  that  the  project  communicadons  not  exist  outside  of  the  context  of  the 
learning  by  the  students.  A secondary  role  of  the  teacher  was  to  acquaint  and  assist  the  students  with  necessary  skills 
for  composing  and  sending  e-mail  messages.  This  included  proof  reading  student  composed  messages  to  insure  that 
they  would  be  understood  and  not  be  redundant.  The  teacher’s  role  in^uded  being  a gate  keeper  for  the  incoming 
messages  to  insure  that  they  were  developmentally  appropriate  and  ch  -irly  stated  for  the  second  graders  to  understand. 

Project  Assessment 

In  assessing  the  benefits  of  the  project,  the  second  grade  teacher  and  senior  citizens  were  interviewed  to 
assess  the  perceived  worth  of  the  experience.  Project  assessment  areas  that  were  intended  to  be  explored  included: 

• appropriateness  for  learning/teaching  class  concepts 

• perceived  worth  of  learning  about  other  group 

• perceived  worth  in  building  relationships  with  other  group 

• perceived  building  of  mutual  respect 

• perceived  similarities  with  other  group 

• perceived  potential  for  long  term  and  on-going  relationships 

• project  implementation  effectiveness  and  weaknesses 

The  second  part  of  the  project  assessment  featured  the  communication  archives.  The  electronic 
communication  between  the  senior  citizens  and  second  graders  have  been  collected  and  a content  analysis  will  be 
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conducted  focusing  on  areas  and  levels  of  engagement  between  the  two  groups  as  well  as  to  provide  a reference  for 
some  of  the  issues  that  emerge  from  the  participant  interviews. 

The  final  component  of  the  project  assessment  featured  observations  of  the  two  groups  in  face  to  face 
communication  in  the  form  of  two  social  “get-togethers”  (Valentine’s  Day  party,  and  an  end-of-the-year  park  outing). 
The  observations  centered  on  how  the  two  groups  demonstrated  ease  in  communicating  with  one  another,  respect  for 
one  another,  and  social  cohesiveness. 


Communication  Content  Analysis 

The  communication  between  the  two  members  were  categorized  in  the  themes  selected  by  the  teacher.  For 
the  year,  these  themes  reflected  common  experiences  that  both  the  second  graders  and  seniors  could  easily  relate  to 
and  reflect  upon.  The  communication  themes  were  as  follows: 

• Reflections  on  celebrating  Halloween 

• Traveling  for  Holidays 

• Reflections  on  celebrating  Thanksgiving 

• Christmas  Traditions 

• Games  that  people  play  and  watch 

• Planning  of  a Valentines  Day  party 

• Hobbies,  interests,  and  collectibles 

• Favorite  books  to  read 

• Favorite  places  to  visit 

• Ways  of  celebrating  Earth-Day 

• Summer  Plans 

It  became  apparent  through  reading  the  messages  that  the  second  graders’  role  of  asking  questions  and  the 
seniors’  role  in  supplying  recollections  became  the  predominant  structure  of  the  communication.  It  was  consistent 
throughout  the  themes  that  the  students  were  interested  in  personal  stories  such  as,  “what  was  it  like  to  go  to  school 
back  then?”  It  became  clear  through  reading  student  composed  that  the  students  were  identifying  with  the  seniors  on 
a more  than  superficial  nature  as  the  project  progressed.  In  one  of  the  messages,  the  class  reported  to  the  seniors 
how  wonderful  it  was  to  talk  with  them  because  most  of  their  real  grandparents  lived  far  away.  This  included  a 
survey  the  students  conducted  concerning  how  many  of  their  grandparents  lived  far  away.  The  results  showed  that  8 
of  the  14  students  had  grandparents  who  lived  outside  of  the  Western  Virginia  area,  10  of  the  14  had  students  living 
outside  of  the  city  of  Blacksburg.  The  opportunity  for  the  students  to  interact  with  grandparents  were  severely 
limited,  thus,  the  role  of  the  seniors  became  more  of  a surrogate  electronic  “grandparent”.  The  messages  reflected 
both  the  students  and  the  seniors  expressing  themselves  in  what  could  be  interpreted  as  a grandparent/grandchild 
reladonship. 

Synopsis  Of  Senior  Responses  To  The  Project 

Initially  some  of  the  seniors’  responses  to  the  teacher’s  postings  were  quite  lengthy  (up  to  five  long 
paragraphs).  The  teacher  soon  found  that  she  had  trouble  keeping  up  with  reading  the  responses  to  the  students,  and 
asked  the  seniors  to  send  responses  which  were  about  two  paragraphs  in  length.  This  proved  to  be  less  intimidating 
to  the  slower  typists  among  the  seniors  and  led  to  more  responses  from  them.  The  seniors  responded  consistently 
whatever  the  topic  themes  turned  out  to  be.  Generally,  if  they  had  no  experience  which  directly  related  to  a theme, 
they  took  the  opportunity  to  tell  stories  about  similar  events  or  activities,  either  from  their  youth  or  from  more 
recent  events  related  to  their  own  family’s  experience.  In  either  case,  their  narrative  was  typically  focused  on 
particular  experiences  in  which  there  was  an  explicit  moral  lesson  to  be  drawn,  essentially  a form  of  fable.  Again, 
this  fits  with  Erikson’s  eighth  stage,  where  seniors  feel  the  need  to  communicate  information  to  their  wider 
community  for  the  purpose  of  passing  on  their  wisdom.  It  is  also  possible  that  this  way  of  communicating  might 
have  been  a product  of  the  fact  that  the  seniors  knew  that  their  peers  would  be  reading  their  responses.  Interestingly 
some  seniors  chose  to  respond  directly  to  the  teacher  and  did  not  post  their  responses  on  the  listserv.  This  may  have 
been  partly  a product  of  the  technology,  but  it  is  clear  that  some  were  not  comfortable  with  making  these  stories 
public. 
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This  reticence  is  also  reflected  in  the  seniors’  responses  when  they  were  asked  about  their  role  in  the  project 
directly.  A typical  response  was  that  the  individual  was  taking  part  in  the  project  for  the  students’  benefit  and  not  for 
their  own,  even  though  it  was  clear  when  they  were  interacting  with  the  students  they  were  enjoying  themselves 
tremendously.  This  was  not  only  because  the  interaction  itself  was  enjoyable,  but  also  because  it  provided  an 
opportunity  to  reflect  upon  their  own  history  and  they  had  a strong  sense  of  creating  a legacy  in  their  community. 

Synopsis  Of  2nd  f-r^ue  Teacher  Response  To  The  Project 

The  teacher  was  very  positive  about  the  project.  The  major  reason  for  this  was  that  it  fit  with  her  regular 
classroom  objectives.  Her  students  were  able  to  use  the  internet  collaboratively  in  authentic  tasks  that  improved 
both  their  reading  and  writing  skills.  The  teacher’s  ability  to  archive  all  the  messages  gave  her  and  the  students  a 
long-term  resource  from  which  to  draw  that  could  be  related  to  any  class  activity.  The  project  brought  new 
volunteers  into  the  school  as  a number  of  the  seniors  came  in  on  their  own  initiative  to  take  part  in  class  and  school 
activities.  This  also  generated  new  allies  in  the  larger  community,  who  are  important  to  the  level  of  support  the 
school  receives.  There  was  also  a personal  benefit  to  the  teacher  as  very  positive  publicity  was  attached  to  the 
school  and  her  class  in  particular.  The  teacher  also  felt  that  the  technology  was  particularly  appropriate  to  the 
project.  When  asked  whether  other  options  (such  as  writing  letters)  would  have  worked  as  well,  her  response  was 
“No,  I have  done  a number  of  projects  with  letters.  Children  do  not  have  that  long  of  an  attention  span.  [By  the 
time  that  you  get  a letter  back]  so  much  time  has  passed  that  you  forgot  what  you  said.” 

Observation  Of  Face  To  Face  Meeting 

The  result  of  the  e-mail  interaction  was  that  when  the  seniors  and  students  finally  met,  they  already  knew 
each  other  and  each  other’s  interests.  By  giving  both  groups  the  opportunity  to  get  to  know  each  other,  potentially 
negative  stereotypes  were  dissolved  before  they  actually  met.  The  internet  had  provided  an  ideal  mechanism  to 
prepare  the  way  for  very  successful  face  to  face  interaction.  Nevertheless,  there  was  some  awkwardness  at  the  first 
meeting  primarily  because  the  seniors  were  unfamiliar  with  the  physical  environment,  but  this  quickly  dissipated, 
and  was  hardly  evident  at  the  second  meeting. 

The  face  to  face  meetings  were  a key  component  of  this  whole  project  for  a number  of  reasons.  Firstly, 
they  provided  an  opportunity  to  get  to  know  the  person  behind  the  experiences  the  students  had  learned  about. 
Secondly,  they  enabled  the  students  to  get  a more  concrete  understanding  of  what  a “senior”  was.  Thirdly,  they 
brought  the  seniors  physically  into  the  school.  The  first  and  third  of  these  were  particularly  significant,  because  on 
the  basis  of  these  personal  relationships  and  being  brought  into  the  school,  a number  of  the  seniors  became 
volunteers  and  active  supporters  of  the  school. 

Discussion 

There  are  a number  of  key  findings  in  this  project.  Although  all  of  the  issues  included  in  the  project 
assessment  plan  have  not  been  included  here,  on-going  assessment  activities  are  continuing  to  be  explored.  One  of 
the  major  goals  for  success  was  to  see  if  the  communication  would  continue  throughout  the  school  year.  It 
succeeded  because  we  negotiated  in  advance  with  both  stockholders,  i.e.  the  teacher  and  the  seniors.  As  a result  the 
project  fit  with  the  teacher’s  agenda,  and  with  the  seniors’  workload.  Secondly,  the  success  of  the  face  to  face 
meetings  (a  crucial  component)  were  successful  and  more  effective  because  of  the  prior  connections  made  through  e~ 
mail.  Thirdly,  the  project  generated  goodwill  toward  the  school  in  a community  that  traditionally  has  been  generally 
not  supportive,  uninterested  and  unaware  of  school  issues. 

Partly  for  this  reason,  some  of  the  other  issues  we  intend  to  explore  include  how  we  can  maintain  seniors’ 
enthusiasm  as  the  project  repeats  from  year  to  year.  We  also  hope  to  address  more  directly  the  question  of  the 
seniors  involvement  and  support  of  the  school.  In  doing  so,  we  hope  to  uncover  the  extent  to  which  the  seniors 
have  an  enhanced  sense  of  belonging  to  their  own  wider  community.  Other  directions  to  our  research  include  trying 
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similar  projects  in  a modified  form  outside  oF  BEV  in  communities  which  do  not  have  the  same  level  of  computer 
ownership  or  internet  connectivity. 

Finally,  this  was  the  first  phase  of  this  project,  designed  in  part  as  a pilot  for  other  projects.  The 
researchers  did  not  know  if  the  project  would  even  last  the  year  , but  the  project  was  successful  from  both  the 
teacher’s  perspective  and  the  seniors’  perspective.  Both  groups  were  very  interested  in  continuing  the  project,  and  it 
has  continued  in  a slightly  modified  form,  which  the  researchers  intend  to  report  on  at  a later  date. 

“Every  time  an  old  person  dies,  a library  closes” 

Regardless  of  whether  this  particular  project  had  succeeded  or  not,  we  would  have  pursued  this  line  of 
inquiry.  Senior  citizens  are  valuable,  but  too  often  ignored  resources  in  our  communities.  The  internet  is  a tool 
which  can  create  opportunities  to  change  that  concept.  Seniors  can  bring  rich,  lived  experience  into  the  classroom, 
and  students  can  learn  more  about  particular  content,  but  they  also  can  learn  to  appreciate  their  own  history  and  that 
of  their  wider  community. 
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Systematic  behavior  observation  is  one  method  of  measuring  children's  behavior  in  ecological  environments 
and  studying  how  their  behavior  compares  to  that  of  other  children  in  the  same  situation  or  how  it  changes 
subsequent  to  intervention  (Shores,  Jack,  Gunter,  Ellis,  DeBriere,  & Wehbv,  1993;  Walker  & Hops,  1976).  The 
main  strength  of  a systematic  behavior  observation  procedure  is  the  objectivity  it  brings  to  the  description  of 
children's  behavior.  Whether  observing  in  clinical  settings,  classrooms,  or  playgroup  situations,  service  providers  in 
psychiatric  and  pediatric  services  find  behavioral  observation  to  be  a critical  skill  for  gathering  reliable  and  objective 
data  to  aid  in  treatment  decisions  (Loney,  1980;  Walker,  Colvin,  Sc  Ramsey,  1995;  Wehby,  Symons,  & Shores, 
1995). 


To  become  a competent  observer,  one  must  learn  a behavior  code  and  procedures,  practice  in  simulated  and  real 
settings  until  proficient,  meet  established  standards  for  reliability,  and  learn  to  interpret  observation  data.  These  skills 
take  an  investment  in  training  and  practice  time.  Typically,  observers  are  trained  through  a combination  of  didactic 
instruction,  work  with  videotapes,  guided  practice  with  a "master"  coder,  and  "in  situ"  reliability  checks.  This 
training  protocol  is  time  and  person  intensive;  it  can  take  40-60  hours  to  bring  a novice  observer  to  the  minimal 
acceptable  level  of  80%  accuracy.  It  is  not  feasible  to  spend  this  amount  of  training  time  on  one  assessment 
procedure  within  most  health  care  training  programs.  As  a result,  full  training  is  rarely  provided  health  care 
personnel,  and  observational  data  are  based  on  subjecdve,  informal  procedures. 

An  alternative  delivery  approach  for  training  personnel  in  observation  skills  is  interactive  multimedia 
technology.  In  the  Classroom  Behavior  Record  (CBR)  Observation  Training  Program,  video  and  audio  scenes  of 
children  in  classrooms/playgroups  and  instructional  narrations  are  stored  on  three  videodiscs.  The  seven-module 
computer  program  delivers  instruction  through  text,  graphics,  animation,  and  videodisc  material;  provides  feedback 
on  practice  activities;  and  controls  delivery  speeds.  The  CBR  instructional  program  is  based  on  a well-established 
observation  system  used  to  assist  in  diagnosis  and  treatment  monitoring  in  hospital  and  school  settings. 

Description  of  the  Classroom  Behavior  Record  Observation  Procedure 

CBR  was  developed  at  The  University  of  Iowa  Hospitals  and  Clinics  by  the  educational  staff  in  the  Child 
Psychiatry  Service  unit  (Nichols,  Robinson,  & Fitzgerald,  1979).  Its  purpose  is  to  provide  a method  of  systematic 
observation  to  document  children's  behaviors  in  naturalistic  settings  for  use  in  diagnosis  and  treatment.  The  original 
CBR  code  was  comprised  of  38  positive  and  negative  behavior  codes  to  provide  a detailed  and  specific  description  of 
behavior.  The  coding  procedure  involves  coding  the  behavior  of  a target  child  in  six-second  intervals,  while  coding 
the  behavior  of  peers  in  the  same  situations  on  alternating  six-second  intervals.  Thus,  the  data  provide  a comparison 
of  the  target  child  to  his/her  peers  under  the  same  situational  expectations.  Results  provide  a clinical  method  to 
evaluate  treatment  effects  through  the  use  of  normative  peer  data  (Walker  & Hops,  1976).  CBR  has  been  used 
continuously  since  1979  in  the  Child  Psychiatry  Service  unit.  Primarily  funded  by  Chapter  I funds  to  the  inpatient 
school  program,  trained  professional  observers  gather  pre-and  post-data  for  children  referred  to  the  treatment  center. 
These  data  are  used  to  understand  the  youngster's  difficulties  as  they  occur  in  natural  settings,  to  establish  treatment 
goals,  and  to  monitor  treatment  effects,  particularly  of  medication.  Training  of  professionals  typically  took  40  60 
hours  of  one-on-one  training  with  periodic  re-calibration  training  and  checking  of  observer  reliability. 

When  the  project  developers  sought  to  implement  training  in  CBR  for  behavior  specialists  in  educational 
settings,  the  original  CBR  code  was  found  to  be  too  complex  for  mastery  and  maintenance  by  school  professionals. 
Through  factor  analysis  procedures  using  hundreds  of  clinical  CBR  records,  the  categories  were  collapsed  into  15 
behavioral  codes  with  4 open  variables  for  tracking  unusual  behaviors  as  defined  by  the  observer  (Fitzgerald,  Nichols, 
& Whittaker,  1992).  The  CBR-School  Version  Code  is  now  implemented  in  various  school  districts  and  provides  the 
content  for  the  current  interactive  multimedia  trying  program  (Fitzgerald,  Semrau,  Nichols,  & Nichols,  1997).  The 
original  coding  procedures  continue  to  be  implemented  with  the  new  code.  Target  and  peers  are  coded  in  six-second 
intervals  on  alternating  intervals.  Data  records  are  used  in  the  special  education  identification  procedure  and  by 
behavioral  specialists  to  monitor  the  effectiveness  of  educational  interventions  for  children.  The  code  has  been 
modified  by  professionals  who  choose  to  look  more  closely  at  the  behaviors  of  special  groups  of  youngsters,  such  as 
early  childhood  groups  or  youngsters  with  autism. 
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The  codes  in  the  CBR-School  Version  are: 


Positive  I ihavior  Codes 

AT  Attend/  On-task 

IM  Incidental  Motor 

II  Instructional  Interaction 

PP  Positive  with  Peer 

PT  Positive  with  Teacher 

CO  Comply 

A G Approval  Received 

V1-V2  Open  Positive  Variables 


Negative  Behavior  Codes 

F A Fail  to  Attend/  Off-task 

PL  Play  with  Object 

M N Motor/  Noise  Obtrusive 

DD  Disruptive,  Destructive 

N P Negative  with  Peer 

N T Negative  with  Teacher 

F C Fail  to  Comply 

D G Disapproval  Gained 

V3-V4  Open  Negative  Variables 


Making  and  Re-purposing  the  Videodisc 

The  initial  videodisc  was  created  by  a production  team  at  The  University  of  Iowa  Hospitals  and  Clinics  with 
combined  funding  and  support  from  tiie  Iowa  State  Department  of  Public  Instruction.  The  University  of  Iowa  WEEG 
Computing  Center,  The  University  of  Iowa  Audio-Visual  Department,  and  The  University  of  Iowa  Hospitals  and  C!  inics 
(Nichols,  Huntley,  Lindamann,  Schiele,  Johnson,  & Fitzgerald,  1985).  Most  of  the  scenes  which  were  taped  for  use  on 
the  videodisc  were  authentic.  Youngsters  were  taped  in  classroom  and  playgroup  situations  in  a child  psychiatric 
inpatient  center  over  a period  of  months  to  gain  as  wide  a variety  of  children  as  possible.  To  fully  capture  good  teaching 
samples  of  behaviors,  six  child  actors  were  recruited  to  demonstrate  some  of  the  critical  behaviors  needing  to  be  taught 
for  CBR. 


When  the  CBR  Code  was  revised  to  the  CBR-School  Version  Code,  the  entire  videodisc  was  re-purposed  to 
teach  this  new  code.  Additional  classroom  video  material  was  secured  from  video  archives  produced  through  a special 
project  grant  from  the  U.S.  Department  of  Education  (Semrau  & Fitzgerald,  1992).  Additional  practice  scenes  were 
chosen  from  these  archives  and  an  additional  videodisc  was  made  to  increase  the  number  of  practice-for-generalization 
opportunities.  The  current  training  program  utilizes  one  side  of  a videodisc  for  the  acquisition  and  fluency-building 
modules  and  two  additional  sides  of  videodiscs  for  the  generalization  practice  modules  and  reliability  testing. 

Instructional  Design  of  the  CBR  Training  Program 

The  overall  design  of  the  training  program  is  based  on  the  Stages  of  Learning  Model  as  initially  described  by 
Gagne  ( 1974)  and  adapted  to  computer-based  instruction  (CB1)  by  Criswell,  1989.  Learning  is  viewed  as  a systematic, 
developmental  process  with  the  learner  progressing  through  stages  of  acquisition,  fluency,  generalization,  and 
maintenance.  Anderson  (1980)  describes  three  stages  in  skill  acquisition:  1)  a cognitive  stage  to  leam  about  the  skill, 
2)  an  associative  stage  in  which  a learner  practices  the  skill,  and  3)  an  autonomous  stage  in  which  a learner  improves 
the  skill  performance. 

These  models  are  very  helpful  for  determining  an  effective  instructional  design  for  skill-based  instruction. 
Criswell  (1989)  posits  that  effective  instruction  requires  a match  between  C3  I design  and  learner  objectives  as  reflected 
in  progressive,  changing  performance.  Effective  instructional  design  should  facilitate  the  mastery  of  learning  objectives 
at  each  stage  in  the  hierarchy.  The  teaching  approach  at  each  stage  must  change  to  support  differing  goals  for  the 
learning.  The  schema  displayed  in  Figure  1 depicts  the  organization  of  the  CBR  training  program.  Design  of  the 
module?  within  the  CBR  program  have  differing  design  features  to  support  acquisition,  fluency-building,  and 
generalization  of  skills  in  behavioral  observation. 

Despite  an  overall  high  level  of  organization  and  structure,  the  modules  in  the  training  program  can  be 
accessed  in  a nonlinear  manner  to  meet  different  learner  preferences.  Movement  across  learning  activities  is  easily 
executed  by  opening  different  modules.  To  facilitate  learner  control  within  the  modules,  the  program  frequently  stops 
and  allows  the  learner  to  repeat  segments,  advance  or  restart  drills,  and  check  scores.  In  the  tutorial  modules,  all 
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subjects  are  listed  on  a pull-down  menu  which  can  be  accessed  at  any  time.  Advisement  for  the  learner  is  inserted 
into  feedback  on  the  drills  which  suggest  levels  of  proficiency  for  each  fluency  drill.  In  early  studies  with  this 
program,  differences  were  observed  in  users'  sequences,  although  most  users  follow  the  organizational  pattern  and 
embedded  structure  within  the  training  program  (Fitzgerald,  1.995). 


Classroom  Behavior  Record 


Routines 

Figure  1 . Matching  CBR  Training  Modules  to  the  Stages  of  Learning  Model 


Acquisition 

The  learner's  instructional  goal  at  the  acquisition  level  is  Vo  acquire  new  knowledge  or  skills.  The  CBR 
training  program  includes  a series  of  tutorials  with  short  practices  embedded  within  each  concept  or  skill.  The 
teaching  hierarchy  is  simple  to  complex,  based  on  the  structure  of  the  knowledge  and  skills  needed  in  this 
observation  system  (Kelly,  Orgel,  & Baer,  1985).  These  procedural  tutorials  provide  information  through  narration, 
screen  notes,  video  demonstrations,  and  guided  practice  segments  with  feedback. 

In  the  Observation  Procedures  module,  tutorials  are  used  to  explain  the  importance  of  gathering  objective 
observation  data,  studying  behavior  within  contexts,  using  the  open  variables  to  define  behaviors  of  special  interest, 
following  a hierarchy  of  rules  for  making  coding  decisions,  and  coding  the  child  and  peers'  behaviors  on  alternating 
intervals.  A four-minute  practice  is  provided  for  the  user  to  apply  the  coding  rules  of  precedence.  The  practice  stops 
at  the  end  of  each  minute  to  allow  the  user  to  enter  his/her  codes  and  compare  entries  to  correct  answers.  Another 
practice  activity  allows  the  user  to  practice  the  alternating  interval  routine  with  a group  of  four  youngsters.  The 
computer  guides  the  user  through  coding  one  student  as  the  target  and  the  other  three  peers  using  the  alternating 
interval  routine.  The  practice  stops  each  minute  to  allow  the  user  to  enter  codes,  see  answers,  and  repeat  if  desired. 
See  Figure  2 for  an  example  of  a tutorial  screen  in  the  Observation  Procedures  module. 

In  the  CBR  Codes  module,  the  behavioral  codes  are  taught  in  four  groups  of  related  behaviors:  task  related 
behaviors,  interactive  behaviors,  compliance,  and  student  as  object.  For  each  code,  the  behavior  is  defined  an  three 
video  examples  are  shown.  In  addition,  brief  practices  are  available  within  each  category  for  practice  different,  bng 
and  using  the  codes.  When  errors  are  made,  feedback  is  provided  to  explain  the  critical  attributes  of  each  code,  luese 
practices  are  untimed  since  they  focus  on  mastery  of  the  codes,  not  speed  of  coding.  Figure  3 displays  the  tutorial  and 
practice  choices  available  for  the  group  of  task-related  behaviors. 
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Figure  2.  Tutorial  Screen  for  the  Observation  Routine  Practice 


Figure  3.  Tutorial  Screen  for  Learning  Task-Related  Behavior  Codes 


Fluency 


Following  acquisition,  the  goal  of  instruction  is  for  the  learner  to  use  the  knowledge  and  skills  in  a fluent 
manner  (Criswell,  1989).  New  information  should  become  automatic  so  it  can  be  recalled  without  undue  cognitive 
processing.  The  CBR  training  program  includes  two  components  for  building  fluency. 

In  the  Text  Flashcard  module,  brief  descriptions  of  behavior  are  displayed  in  text  on  the  screen.  The  format  for 
the  text  flashcard  is  displayed  in  Figure  4.  There  are  approximately  100  text  flashcards  which  are  presented  in  random 
order  at  three  controlled  speeds,  allowing  a limited  time  to  read  the  description  and  enter  the  user's  code  into  the 
computer  keyboard.  The  correct  code  is  given  and  explained  if  the  user  enters  an  incorrect  code  or  if  time  runs  out 
before  the  code  is  entered.  A running  score  is  provided  the  user  along  with  suggested  levels  of  achievement  for 
increasing  the  practice  speed.  The  goal  is  to  obtain  fluency  at  the  six-second  speed  needed  for  actual  behavior  coding 
with  the  CBR  procedure.  Rapid  responding  to  these  flashcards  builds  automaticity  in  remembering  the  codes. 
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Figure  4.  Text  Flashcard  Drill  Card 

The  second  fluency-building  module,  Video  Flashcards , provides  a more  realistic  coding  simulation  because  it 
uses  actual  video  scenes  of  children.  In  addition  to  continuing  to  build  speed  and  accuracy,  the  fidelity  of  the  practice 
is  increased  in  this  phase  of  instruction  (Alessi,  1983).  The  format  for  the  video  flashcard  is  displayed  in  Figure  5. 
The  flashcard  module  has  100  brief  video  clips  displaying  classroom  behaviors  which  are  displayed  in  random  order. 
The  user  is  allowed  3 seconds  to  enter  his/her  code  following  the  video  segment.  The  correct  code  is  given  and 
explained  if  the  user  enters  an  incorrect  code  or  if  time  runs  out  before  the  code  is  entered.  A running  score  is 
provided  the  user  along  with  a suggested  level  of  achievement  to  have  mastered  the  fluency  module  of  the  training. 
Through  these  video  flashcards,  users  increase  their  ability  to  rapidly  interpret  and  code  behaviors  youngsters' 
behaviors  in  a wide  variety  of  situations. 


Figure  5.  Video  Flashcard  Drill  Card 
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Generalization  and  Maintenance 


Goals  at  the  generalization  stage  are  for  the  learner  to  apply  knowledge  or  skills  in  the  way  they  will  be  used 
"in  situ"  and  to  practice  to  mastery.  The  practice  situations  must  be  similar  to  those  used  during  instruction  but  offer 
mixed  practice  with  new,  realistic  scenarios  (Kelly  et  alM  1985).  Classroom  and  playgroup  scenes  are  included  to 
provide  extensive  practice  for  the  learner  which  simulates  tne  job  of  an  observer  in  psychiatric  and  educational 
settings. 

There  are  a total  of  eight  scenes  in  the  Group  Practices  module  providing  practice  opportunities  ;th  s total  of 
18  different  youngsters  varying  from  ages  5 to  15.  The  scenes  show  youngsters  in  play  group  and  classroom 
situations.  These  practices  follow  the  actual  CBR  procedures:  the  user  alternately  codes  the  target  student  and  peers 
with  a continuous  six- second  interval  and  marks  his/her  codes  on  a paper/pencil  recording  form.  Each  segment  runs 
for  10  intervals  (1  minute)  and  then  pauses  to  allow  the  user  to  enter  his/her  codes  and  compare  codes  to  those  of  the 
master  coder  to  discern  differences  and  compute  reliability  based  on  the  10  intervals.  The  user  may  choose  to  repeat 
each  minute  for  review  or  continue  forward.  Figure  6 displays  the  format  for  the  data  entry  and  comparison.  In  this 
example,  Mark  is  the  target  child  and  BJ  and  Ky  are  peers. 


Figure  6.  Data  Entry  Screen  for  Classroom  Application 

The  CBR  program  includes  two  methods  for  testing  mastery  of  coding  and  reliability.  In  the  Codes  Test 
module,  30  video  flashcards  are  displayed  for  six-seconds  each  and  the  user  marks  the  code  on  a paper/pencil  recording 
sheet.  This  test  is  designed  to  test  accuracy  in  coding  and  allows  the  user  adequate  time  to  think  and  enter  his/her 
codes  into  the  computer.  Approximately  half  the  video  segments  are  new  to  the  user  as  they  were  withheld  from 
prior  practice.  Results  are  provided  the  user  following  the  test. 

The  second  testing  module,  Reliability  Tests , operates  the  same  as  the  Group  Practices.  Four  youngsters  are 
displayed  in  the  video,  and  each  may  be  selected  as  the  target  child;  thus,  four  reliability  tests  can  be  run  in  this 
module  to  check  mastery  and  maintenance  over  time.  The  test  is  five  minutes  long  (50  intervals)  and  the  user  codes 
target  and  peers  on  alternating  intervals,  marking  the  codes  on  a paper/pencil  coding  sheet.  When  the  test  is 
completed,  the  user  enters  his/her  codes  into  the  computer  program  and  receives  a reliability  score. 
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Methodology  and  Outcomes 


Subjects  and  Training  Procedure 

The  CBk  training  program  has  been  implemented  with  40  graduate  students  enrolled  in  special  education 
courses  at  a mid- Atlantic  university.  These  students  were  enrolled  either  in  teacher  certification  programs  in  learning 
and  behavioral  disorders  or  in  the  doctoral  program  in  special  education.  Data  were  collected  over  a three-year  time 
period  utilizing  multiple  instructors. 

Following  a general  demonstration  of  the  CBR  training  materials  and  oveiview  of  the  coding  categories,  25  of 
the  users  proceeded  independently  through  all  modules  of  the  training  progrrm.  For  15  of  the  users,  a course 
instructor  utilized  the  acquisition  modules  with  the  class  as  a whole  prior  to  their  independent  use  of  the  program. 
Proficiency  tests  were  scheduled  approximately  5-6  weeks  after  independent  practice  started.  If  required  reliability 
levels  were  not  achieved  at  the  initial  test  session,  additional  practice  and  re-testing  were  required  until  reliability 
standards  were  met. 

The  data  reported  below  are  the  outcome  scores  for  each  user  on  his/her  first  attempt  on  the  outcome  tests. 
Users  were  not  certified  for  using  the  code  and  procedure  in  actual  educational  or  clinical  practice  until  the 
certification  criteria  were  met.  Certification  requirements  were  met  by  one  of  two  score  combinations:  72%  on  codes 
test  and  80%  on  reliability  test,  or  80%  on  codes  test  and  78%  on  reliability  test.  With  extended  practice,  all  but  two 
users  met  criteria  to  be  certified  in  CBR  for  actual  use  with  children. 

Training  Outcomes 

Usage  Time.  The  average  length  of  program  engagement  time  for  users  to  pass  reliability  testing  equalled 
15.68  hours.  The  range  extended  from  5.18  hours  to  36  hours  of  practice  time.  These  data  do  not  include  off-line 
preparation  time  students  may  have  spent  studying  the  codes. 

Learning  Patterns.  Averaging  across  all  40  users,  approximately  30%  of  on-line  training  time  was  spent  in 
the  acquisition  stage  of  learning  the  code;  25%  of  on-line  training  time  was  spent  in  the  fluency-building  stage  of 
learning;  and  45%  of  on-line  training  time  was  spent  in  the  generalization  stage  with  simulated  practice. 

Outcomes  on  the  Codes  Test.  The  Codes  Test  measures  knowledge  of  the  codes.  The  mean  score  for  the 
users  equalled  76.9%  (s.d.  = 8.875)  on  their  first  attempt  on  the  test. 

Outcomes  on  the  Reliability  Test.  The  Reliability  Test  measures  both  speed  and  accuracy  in  coding.  The 
mean  score  for  the  users  equalled  80. 125%  (s.d.  = 6.086)  on  their  first  attempt  on  the  test. 

Summary 

The  Classroom  Behavior  Record  Observation  Training  Program  appears  to  be  an  effective  and  efficient 
training  procedure  for  such  a comprehensive  code  and  complex  observation  procedure.  Comparing  the  training  time 
to  records  established  at  The  University  of  Iowa  Hospitals  and  Clinics  with  the  original  38-item  code,  training  time 
has  been  reduced  from  40-60  hours  to  16  hours.  Some  of  this  training  time  difference  may  be  related  to  the  reduced 
complexity  of  the  CBR-School  Version  Code  with  16  categories  in  comparison  to  the  38  categories  in  CBR. 
However,  it  is  noteworthy  that  these  training  results  were  achieved  with  minimal  direct  training  from  an  instructor  or 
a master  coder  sitting  side-by-side  with  a novice  observer  for  extensive  practice  using  videotapes  or  live  classrooms. 

The  instructional  design  of  the  training  program,  the  authenticity  of  the  materials,  and  the  program's  ability  to 
provide  learner-controlled  instruction  are  important  ingredients  in  the  effectiveness  of  this  interactive,  multimedia 
training  program.  While  the  stages  of  learning  model  provided  the  overall  structure  for  designing  the  training 
activities,  its  learner-centered,  hypermedia  interface  allows  users  to  select  their  own  paths  through  the  stages  of 
acquisition,  fluency,  and  generalization  and  move  from  cognition  to  autonomous  skill  performance. 
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Dissemination 

As  provided  by  grant  #H029K30210  from  the  U.S.  Department  of  Education  grant,  copies  of  the  CBR 
training  program  are  being  distributed  to  training  institutions,  state  departments  of  education,  and  inservice  agencies. 
The  materials  are  available  for  both  the  Macintosh  and  Windows  platforms.  Needed  equipment,  in  addition  to  a 
computer  system,  includes  a Pioneer,  Panasonic,  or  Sony  videodisc  player  with  monitor  and  connecting  cable.  The 
only  cost  for  the  materials  is  a postage  and  handling  charge.  Eligible  agencies  must  file  an  implementation  plan  for 
training  initiatives.  Complete  information  on  the  programs  is  available  by  contacting  the  authors  or  through  the 
project  website  at:  http://tiger.coe.missoun.edu/tpss.html. 
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An  Implementation  Model  for  Integrated  Learning  Systems 
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ABSTRACT 

Integrated  learning  systems  (ILSs)  provide  a multi-year  curriculum  sequence  of  computer-based  instruction 
controlled  by  a sophisticated  management  system.  ILS  implementation  is  better  understood  when  the  focus  of 
implementation  is  shifted  from  the  technology  to  the  people  who  use  the  technology;.  This  study  examined  the 
operational  patterns  of  teachers  implementing  an  ILS  in  elementary  schools  in  a metropolitan  school  district.  A 
measure  was  developed  that  described  the  major  components  of  i'mple7 mentation  of  ILS  technology.  Data  regarding 
the  operational  patterns  of  teachers  using  ILS  technology  were  collected  from  teacher  interviews  and  analyzed.  The 
best  ILS  implementation  practices  included  integration  with  classroom  instruction,  training  in  the  use  of  an  ILS,  and 
the  use  of  motivational  strategies.  Implementation  is  better  understood  when  contextual  phenomena  of  the 
implementation  are  examined  because  the  question  of  the  instructional  effectiveness  of  an  ILS  or  other  forms  of 
computer-delivered  instruction  may  be  as  much  an  implementation  issue  as  a matter  of  instructional  design. 

Introduction  and  Background  of  Study 

To  suppose  that  users  of  computer-delivered  instruction  implement  courseware  in  the  way  it  is  intended  to 
be  used  is  often  an  ambitious  assumption.  Smith  and  Ragan  (1993)  advised  that  "in  drawing  the  line  of  causation 
from  the  instruction  to  the  results,  it  is  critical  to  be  able  to  identity  the  degree  to  which  the  description  of  the 
program  represents  what  actually  occurred  during  instruction  with  the  new  program"  (p.  416).  Therefore,  the 
determination  of  effectiveness  of  computer-delivered  instruction  needs  to  begin  with  a determination  of  the  degree  of 
implementation  of  the  instruction. 

This  article  describes  the  development,  validation,  and  research  application  of  an  instrument  for  evaluating 
the  implementation  of  computer-delivered  instruction.  This  instrument,  the  Computer-Delivered  Instruction 
Configuration  Matrix  (CDICM),  was  designed  to  examine  the  implementation  of  Integrated  Learning  Systems 
(ILSs).  The  instrument  was  validated  in  a study  by  Mills  (1997)  involving  four  metropolitan  elementary  schools.  The 
purpose  of  this  paper  is  to  describe  the  development  and  deployment  of  the  CDICM  and  its  implications  for 
educational  change  and  reform. 

The  CDICM  was  adapted  from  the  Innovation  Configuration  Matrix  (ICM),  a tool  developed  by  Heck, 
Steigelbauer,  Hall,  and  Loucks  (1981)  for  use  in  identifying  the  essential  components  of  an  innovation  and  the 
variations  of  implementation  for  each  of  the  innovation  components.  The  ICM  is  an  effective  resource  for  assessing 
implementation  of  instructional  technology  and  other  educational  innovations  (Albers,  1994;  George  & Hord,  1980; 
G leghorn,  1993;  Hord,  Huling,  & Austin,  1986).  The  ICM  provides  an  understanding  of  the  operational  patterns  of 
users  of  an  innovation  and  therefore  allows  the  evaluators  of  an  innovation  to  make  judgments  about  appropriate 
practices  and  how  much  variation  in  practices  is  acceptable  (Hord,  Rutherford,  Huling-Austin  & Hall,  1987). 

Hall  and  Loucks  (1977)  were  disappointed  when  experimental  research  and  evaluation  studies  presumed  the 
treatment  was  ‘present  and  its  effects  were  accounted  for  by  testing  for  statistically  significant  differences  between 
two  groups  or  by  pre-  and  post-testing  the  same  sample.  They  concluded  that  information  about  the  actual  use  and 
degree  of  Implementation  of  an  innovation  might  better  explain  some  nonsignificant  findings  reported  in  evaluation 
and  experimental  studies.  According  to  Hall  and  Loucks  (1978)  a particular  innovation  can  have  several  different 
operational  patterns  as  a result  of  variations  in  the  selection  and  use  of  innovation  components.  They  labeled  these 
operational  patterns  as  innovation  configurations.  Different  patterns  or  configurations  resulted  when  different 
teachers  put  innovations  into  operation  in  their  classrooms.  Individual  teachers  used  different  components  of  an 
innovation  in  different  ways  and  when  these  components  were  put  together  a number  of  patterns  emerged  that 
characterized  different  uses  of  the  innovation  (Hord,  Rutherford,  Huling-Austin,  & Hall,  1987). 

Development  and  Field  Testing  of  Innovation  Configuration  Matrix 

The  construction  of  the  CDICM  was  based  on  a review  of  research  on  computer-delivered  instruction, 
courseware  documentation,  telephone  interviews  with  courseware  developers,  and  interviews  with  vendor  training 
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facilitators,  lab  managers,  and  teachers  who  used  the  Computer  Curriculum  Corporation  (CCC)  courseware, 
Successmaker . Successmaker  courseware  covers  subjects  ranging  from  math  and  reading  to  GED  preparation. 

Using  ICM  development  techniques  an  ICM  was  developed  to  fit  with  the  adoption  and  use  of  the 
Successmaker  ILS  in  varying  degrees  and  patterns  of  implementation.  The  matrix  consisted  of  a component  checklist 
comprised  of  five  variations  for  each  component  with  each  successive  variation  indicating  a level  of  use  representing 
a closer  approximation  of  ideal  use.  Although  the  CDICM  instrument  was  developed  and  tested  with  a particular 
ILS,  its  application  is  relevant  for  many  forms  of  computer-delivered  instruction. 

The  CDICM  was  devised  as  an  instrument  to  be  completed  by  an  evaluator  based  on  responses  supplied  by 
teachers  during  an  interview.  The  component  uf  technology  use  and  variations  of  the  components  selected  for 
inclusion  on  the  CDICM  were  presumed  to  reflect  the  actual  and  ideal  practices  of  teachers  involved  with  ILS 
implementation.  Although  many  forms  and  variations  of  technology  adoption  and  use  by  teachers  may  exist,  the 
process  of  instrument  development  attempted  to  reflect  a true  and  accurate  gradient  of  technology  use  from  "least 
acceptable"  to  "ideal."  Since  truth  and  accuracy  of  instrumentation  was  not  absolute,  the  intent  was  to  develop  an 
instrument  that  provided  estimates  that  possessed  known  and  reasonable  validity  and  reliability. 

As  part  of  the  development  process,  the  CDICM  was  subjected  to  a two-phase  field  test.  In  the  first  phase 
interviews  were  conducted  with  a sample  of  ten  teachers  who  regularly  used  the  courseware  for  instruction.  The 
purpose  of  this  administration  of  the  CDICM  was  to  make  revisions  to  matrix  components  and  to  make  revisions  to 
the  interview  guide.  After  revisions  were  made,  the  second  phase  of  the  field  test  was  conducted  to  establish 
reliability  and  validity  for  the  final  version  of  the  CDICM.  A sample  of  nineteen  teachers  who  regularly  used  the 
courseware  completed  a CDICM  checklist  using  the  revised  CDICM  and  then  five  of  the  subjects  completing  the 
checklist  were  selected  for  interviews. 

Descriptive  statistics  were  computed  for  total  instrument  reliability.  The  final  15-item  checklist  allowed  for 
a range  of  total  scores  from  0 to  75.  The  standard  error  of  measurement  for  the  field  test  was  3.68  while  the 
int-irscorer  reliability  coefficient  was  .96  for  all  items.  This  administration  of  the  CDICM  rhecklist  yielded  a 
coefficient  alpha  of  .61  (p  < .01)  for  the  total  scale.  The  final  version  of  the  CDICM  checklist  and  interview  guide  is 
provided  at  the  end  of  this  article. 

Data  Collection  and  Analysis  Procedures 


The  final  version  of  the  CDICM  was  deployed  in  a school  district  that  had  made  a substantial  investment  in 
ILS  technology.  ILS  technology  was  used  in  all  elementary-’  school  campuses  in  the  district.  The  school  district 
initially  implemented  ILS  technology  in  six  elementary  schools  and  during  the  previous  school  year  expanded  the 
program  to  include  almost  all  kindergarten  through  eighth  grade  students  in  the  school  district. 

A lab  configuration  was  employed  at  each  school  with  a number  of  workstations  distributed  to  some 
classrooms.  Each  school  had  a lab  coordinator  or  media  specialist  to  coordinate  the  computer  lab  activities  and  to 
assist  the  classroom  teacher  in  the  management  and  coordination  responsibilities  of  the  computer  lab  and  in 
facilitating  ILS  instruction.  The  school  principal  developed  a lab  schedule  in  which  all  students  at  each  school  were 
provided  forty-five  minute  sessions  using  the  ILS  two  or  three  times  per  week  to  supplement  regular  classroom 
instruction. 

The  sample  selected  for  this  study  consisted  of  elementary  school  teachers  at  four  schools  whose  students 
interacted  with  an  ILS  using  the  Successmaker  courseware  for  instruction.  A sample  of  teachers  (N=30)  was 
randomly  selected  for  the  study  from  all  of  the  ILS-using  teachers  in  the  four  schools.  The  schools  selected  for  this 
study  had  employed  ILS  technology  for  different  periods  of  time  and  since  the  sample  size  was  relatively  small,  no 
comparisons  were  made  among  the  various  schools. 

To  collect  data  regarding  patterns  of  use  occurring  among  individuals  implementing  an  ILS  and  differences 
among  these  patterns  of  use,  a structured  interview  was  conducted  and  recorded  by  the  researcher  with  each  of  the 
teachers  selected  for  the  random  sample.  One  interview  in  the  original  selection  was  not  completed  making  N=29  for 
the  CDICM. 

An  expert  group  consisting  of  the  researcher,  an  experienced  lab  manager  (not  from  the  school  system  under 
study),  and  a vendor  representative  was  assembled  to  rate  the  interviews  for  the  CDICM  checklist.  The  CDICM 
component  scores  for  each  case  in  the  sample  were  based  on  the  collective  ratings  of  the  expert  group.  The  expert 
group  reviewed  transcribed  audio  tapes  of  teachers'  interviews  and  independently  scored  each  respondent  based  on 
the  content  and  structure  of  the  CDICM  checklist. 
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Analysis  of  use  by  teachers  implementing  the  ILS  was  conducted  through  pattern  analysis  of  the  interview 
data  to  determine  dominant  configuration  patterns  or  variations.  The  general  patterns  that  emerged  were  analyzed  by 
a cluster  analysis  that  identified  relatively  homogenous  groups  of  cases  based  on  selected  characteristics. 

One-way  analysis  of  variance  was  used  to  examine  differences  among  the  configuration  patterns  for  each  of 
the  components  measured  by  the  CDICM.  A test  was  performed  to  validate  the  results  of  the  ANOVA  against 
discrepancies  that  existed  in  the  assumptions,  due  to  differences  in  group  sizes  and  normality  (Minium,  1978; 
Norusis,  1995).  Post-hoc  tests  were  performed  to  compare  differences  between  pairs  of  configuration  patterns’ 
means  for  each  CDICM  component  using  the  Bonferroni  procedure. 

An  independent  variable  was  constructed  for  CDICM  level  of  use  by  assigning  a level  of  use  classification 
for  the  composite  score  of  the  CDICM  to  each  case  (using  the  same  scale  as  was  used  for  the  component  variations 
on  the  CDICM  (1,  2,  3,  4 or  5).  Next,  a linear  regression  analysis  was  used  to  examine  the  relationship  between  each 
significant  CDICM  component  (from  the  ANOVA)  and  the  degree  of  ILS  implementation.  A stepwise  multiple 
regression  was  performed  to  determine  which  components  were  the  best  predictors  of  ILS  implementation. 

Results  of  the  Data  Analysis 

Reliability  statistics  were  computed  to  determine  the  internal  consistency  in  the  ratings  determined  by  each 
scorer  as  well  as  the  external  consistency  in  ratings  among  the  scorers.  External  consistency  of  the  ratings  among  the 
three  scorers  for  all  items  on  the  scale  was  r = .9926  for  AB,  r ~ .9974  for  AC,  and  r - .9935  for  BC.  Internal 
consistency  was  indicated  by  a significant  coefficient  alpha  (p  < .01)  for  each  scorer  and  for  all  scorers  combined. 

Pattern  analysis  of  the  interview  data  was  performed  to  determine  dominant  configuration  patterns  or 
variations.  This  procedure  identified  relatively  homogenous  groups  of  cases  by  computing  the  centroid  for  each 
dominant  configuration  pattern  that  emerged  from  the  data  and  then  assigning  each  case  to  the  cluster  with  the 
nearest  centroid  until  no  cases  changed  cluster  membership.  Since  the  initial  cluster  centers  and  the  number  of 
dominant  patterns  were  unknown,  this  procedure  consisted  of  selecting  all  fifteen  components  of  the  CDICM  for  use 
in  the  cluster  analysis  and  incrementing  the  number  of  clusters  until  a reasonable  model  was  obtained.  The  number  of 
clusters  started  with  two  and  was  incremented  by  one  until  convergence  of  cluster  centers  reached  a distance  of  zero 
and  the  number  of  cases  in  each  cluster  was  similar.  The  cluster  analysis  was  run  for  two,  three,  and  four  clusters 
before  a reasonable  model  was  selected. 

At  two  clusters  convergence  was  achieved  with  a maximum  distance  of  center  change  of  .3205  while  at 
three  and  four  clusters  convergence  was  achieved  with  a maximum  distance  of  center  change  of  .0000.  At  two 
clusters  the  distance  between  final  clusters  centers,  was  7. 1883  with  18  cases  in  one  cluster  and  1 1 cases  in  the  other. 
However,  at  four  clusters  the  distances  between  final  cluster  centers  ranged  from  7.6997  to  17.1270  and  the  number 
of  cases  in  each  cluster  ranged  from  1 to  16.  The  best  model  occurred  with  the  number  of  clusters  set  at  three.  When 
the  number  of  clusters  was  set  at  three,  convergence  of  cluster  centers  was  achieved  after  four  iterations  and  the 
maximum  distance  by  which  any  cluster  center  changed  was  zero.  When  the  number  of  clusters  was  set  at  three,  the 
number  of  cases  in  each  cluster  ranged  from  8 to  12. 

In  order  to  assess  the  adequacy  of  the  classification  of  implementation  pattern  groups  by  the  cluster  analysis 
a Discriminant  Analysis  (DA)  was  performed.  The  fifteen  CDICM  components  were  used  to  separate  the  groups  into 
the  discriminant  functions.  As  a result  of  this  procedure  all  grouped  cases  were  correctly  classified. 

Table  1 lists  each  CDICM  component  by  F(2,26)  and  the  corresponding  discriminant  function  coefficients 
and  Figure  l provides  a plot  of  the  discriminant  functions  for  each  of  the  CDICM  patterns.  The  discriminant  function 
coefficients  reflected  the  importance  attached  to  each  CDICM  component  in  distinguishing  the  discriminant 
functions.  The  multivariate  analysis  of  variance  performed  by  the  DA  produced  significant  differences  among  the 
Implementation  Patterns  on  CDICM  components  1,  2,  3,  4,  5,  1 1,  and  12  (p  < .05).  Based  on  both  functions  of  the 
DA,  Pattern  2 was  indicative  of  a high  degree  of  implementation  characterized  by  high  training  and  support, 
integration  into  classroom  and  curriculum,  and  use  of  reinforcement  and  motivational  strategies. 

To  examine  differences  among  the  CDICM  configuration  patterns  a one-way  analysis  of  variance 
(ANOVA)  was  performed.  One-way  analysis  of  variance  determined  significant  differences  among  the 
implementation  patterns  groups  for  each  of  the  CDICM  components.  A test  of  homogeneity  of  variance  was 
performed  to  validate  assumptions  required  by  the  ANOVA  and  to  determine  whether  the  variance  of  the  dependent 
variables  was  significantly  different  among  the  implementation  pattern  groups.  For  the  most  part,  the  assumption  of 
equality  of  variances  was  satisfied.  However,  the  Levene  statistic  produced  by  the  homogeneity  of  variance  test  was 
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Table  /.  Discriminant  Function  Weights  for  CDICM  Components 


CDICM  Component 

F 

Sig» 

Fn  I 

Fn  2 

12.  Uses  Reinforcement/Motivational  Strategies 

67.8826 

.0000 

1.25823 

-.30846 

2.  Received  Training  in  Use  of  ILS 

10.2191 

.0005 

.13897 

.29295 

5.  Integrates  with  Classroom  Instruction 

9.2215 

.0009 

-.27445 

1 .74723 

3.  On-Going  Support  is  Provided 

7.6386 

.0025 

.60909 

.18948 

1 1.  Facilitates  ILS  Instruction 

6.5705 

.0049 

.33663 

.14500 

1.  Understands  ILS  Instructional  Design 

3.4095 

.0484 

-.16839 

.28902 

4.  Sets  Instructional  Goals  for  ILS 

2.8557 

.0757 

-.23317 

.61968 

13.  Uses  ILS  Management  Reports 

2.5546 

.0971 

-.36072 

-.09096 

10.  Provides  Sufficient  Time  on  Task 

2.1375 

.1382 

1.48069 

-.53613 

9.  Sets  Clear  Rules  of  ILS  Use 

1.7029 

.2019 

-.15515 

.10001 

6.  Integrates  with  District  Curriculum 

1.5184 

.2379 

.35935 

-.59012 

15.  Understands/Uses  ILS  Routines  and  Equipment 

.9081 

.4157 

-.07324 

.08089 

7.  Individualizes  Enrollment  Options  on  ILS 

.6088 

.5516 

.25277 

-.42373 

8.  Effective  Scheduling  of  ILS 

.4905 

.6179 

-1.07805 

-.77891 

14.  Uses  ILS  Achievement  in  Learner  Evaluation 

.3634 

.6988 

.14913 

-.58905 
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Figure  /.  Plot  of  Discriminant  Functions  for  Each  CDICM  Implementation  Pattern. 

significant  (p  < .05)  for  two  CDICM  components:  Component  7 — Individualizes  Enrollment  Options  on  the  ILS  and 
Component  9 — Sets  Clear  Rules  of  ILS  Use. 

A plot  of  these  two  variables  revealed  that  the  homogeneity  of  variance  assumption  was  violated  by  four 
outlier  cases  for  Component  7 and  three  outlier  cases  on  Component  9.  Several  attempts  to  transform  these  variables 
yielded  no  success  in  obtaining  a nonsignificant  Levene  statistic  and  in  satisfying  the  assumption  of  equality  of 
variances  and  so  these  variables  were  discarded  from  further  analysis. 
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An  ANOVA  examined  the  differences  among  the  implementation  patterns  for  each  of  the  components 
measured  by  the  CDICM  (see  Table  2).  The  observed  power  was  calculated  for  the  Ftest  (p  -.05).  The  results  of  the 
ANOVA  were  consistent  with  the  multivariate  analysis  of  variance  performed  by  the  discriminant  analysis.  The 
ANOVA  indicated  significant  differences  for  six  of  the  CDICM  components. 

To  validate  the  results  of  the  ANOVA  in  regard  to  violations  of  assumptions  about  differences  in  group  sizes 
and  normality,  a Kruskal-Wallis  nonparametric  analysis  of  variance  was  performed.  The  Kruskal-Wallis  test 
confirmed  the  results  of  the  ANOVA  for  five  of  the  six  significant  CDICM  components.  Only  Component  1 — 
Understands  ILS  Instructional  Design  yielded  a significant  difference  on  the  ANOVA  but  not  on  the  PCruskal- Wallis. 

Table  2.  ANOVA  of  CDICM  Interview  Data  Grouped  by  Implementation  Pattern 


Dependent  Variable 

df 

F 

_ Sig. 

Observed 

Power 

1 . Understands  ILS 

Between  Groups 

2 

3.409 

.048 

.589 

Instructional  Design 

Within  Groups 

26 

Total 

28 

2.  Received  Training 

Between  Groups 

2 

10.219 

.001 

.975 

in  Use  of  ILS 

Within  Groups 

26 

Total 

28 

3.  On-Going  Support 

Between  Groups 

2 

7.639 

.002 

.920 

Is  Provided 

Within  Groups 

26 

Total 

28 

4.  Sets  Instructional 

Between  Groups 

2 

2.856 

.076 

man* 

Goals  for  ILS 

Within  Groups 

26 

Total 

28 

■H 

5.  Integrates  with 

Between  Groups 

2 

9.221 

.001 

.961 

Classroom  Instruction 

Within  Groups 

26 

Total 

28 

6.  Integrates  with 

Between  Groups 

2 

1.518 

.238 

.293 

District  Curriculum 

Within  Groups 

26 

Total 

28 

8.  Effective  Scheduling 

Between  Groups 

2 

.491 

.618 

.122 

of  ILS 

Within  Groups 

26 

Total 

28 

10.  Provides  Sufficient 

Between  Groups 

2 

2.137 

.138 

.398 

Time  on  Task 

Within  Groups 

26 

Total 

28 

11.  Facilitates  ILS 

Between  Groups 

2 

6.570 

.005 

.874 

Instruction 

Within  Groups 

26 

Total 

28 

12.  Uses  Reinforcement/ 

Between  Groups 

2 

67.883 

.000 

1.000 

Motivational  Strategies 

Within  Groups 

26 

Total 

28 

13.  Uses  ILS 

Between  Groups 

2 

2.555 

.097 

.465 

Management  Reports 

Within  Groups 

26 

Total 

28 

14.  Uses  ILS  Achievement 

Between  Groups 

2 

.363 

.699 

.102 

in  Learner  Evaluation 

Within  Groups 

26 

Total 

28 

15.  Understands/Uses  ILS 

Between  Groups 

2 

.908 

.416 

.190 

Routines  and  Equipment 

Within  Groups 

26 

1 

Total 

28 
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Post-hoc  Bonferroni  tests  were  used  to  compare  differences  between  pairs  of  configuration  patterns’  means 
for  CDICM  components  yielding  a significant  difference  on  both  the  ANOVA  and  the  Kxuskal-Wallis  (see  Table  3). 
The  Bonferroni  comparisons  indicated  significant  mean  differences  (p  < .05)  for  all  five  CDICM  components  for 
configuration  Pattern  2 when  compared  to  Patterns  1 and  3 except  for  Component  12.  These  post-hoc  findings 
indicated  that  differences  among  the  configuration  patterns  revealed  by  analysis  of  variance  were  primarily  based  on 
differences  between  Pattern  2 and  the  other  two  patterns.  Post-hoc  testing  validated  that  these  differences  were 
significant  for  Components  2,  3,  5,  11,  and  12,  Thus,  teachers  who  implemented  the  ILS  according  to  Pattern  2 were 
statistically  better  implementers  than  teachers  who  implemented  the  ILS  according  to  Patterns  l or  3. 

Table  3.  CDICM  Components  Significant  at  95%  Confidence  Interval  in  Post  Hoc  Comparisons 


■HHH 

Implementation 

Pattern 

Implementation 

Pattern 

Mean 

Difference 

Std. 

Error 

Sig. 

2.  Received 

1 

2 

-3.28* 

1.089 

mEm 

Training  in 

3 

1.31 

.988 

.593 

Use  of  ILS 

2 

1 

3.28* 

1.089 

.017 

3 

4.58* 

1.022 

.000 

3 

i 

-1.31 

.988 

.593 

2 

-4.58* 

1.022 

.000 

3.  On-Going 

1 

2 

-2.39* 

WBM 

.022 

Support  Is 

3 

.53 

■SI 

1.000 

Provided 

2 

1 

.819 

msm 

3 

mm  9 

.769 

■ S 

3 

1 

-.53 

.743 

1.000 

2 

-2.92* 

.769 

.002 

5.  Integrates  with 

1 

2 

mmm 

1.084 

.002 

Classroom 

3 

.984 

1.000 

Instruction 

2 

1 

MEM 1 

mmm 

MM 

3 

fin 

■IBS 

WmMjm 

3 

1 

.56 

.984 

1.000 

2 

-3.71* 

1.018 

.004 

11.  Facilitates  ILS 

1 

2 

-1.76* 

.608 

.022 

Instruction 

-> 

.19 

.552 

1.000 

2 

l 

.608 

MSM 

3 

■ 

.571 

•% 

1 

* v ]* . . y pBFjVR' ■ -y  ’ 

.552 

1.000 

2 

M 

.571 

.006 

12.  Uses 

1 

2 

■ 

.769 

MM 

Reinforcement/ 

1 

.698 

■S 

Motivational 

2 

1 

7.64* 

.769 

.000 

Strategies 

3 

.42 

.722 

1.000 

3 

1 

7.22* 

.698 

.000 

2 

-.42 

.722 

1.000 

*The  mean  difference  is  significant  at  the  .05  level. 


A stepwise  multiple  regression  was  performed  to  determine  significant  CDICM  components  or 
combinations  of  components  that  were  the  beat  predictors  of  the  degree  of  ILS  implementation  (see  Table  4). 
Stepwise  variable  selection  entered  each  CDICM  component  into  the  regression  model  by  order  of  importance  and 
removed  components  with  diminished  Importance  as  additional  predictors  were  added. 

All  CDICM  components  were  entered  into  the  regression  model  as  predictor  variables  and  the  degree  of 
ILS  Implementation  as  measured  by  CDICM  composite  score  was  entered  into  the  regression  model  as  the  criterion 
variable.  Significant  correlations  (p  < .01)  were  obtained  among  all  the  variables.  Component  5 — Integrates  with 
Classroom  Instruction  explained  56%  of  the  variability  in  the  degree  of  ILS  implementation  for  this  model  and  54% 
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for  other  data  sets  from  the  same  population.  Component  5,  Component  2 — Received  Training  in  Use  of  ILS , and 
Component  12 — Uses  Reinforcement/Motivation  Strategies  explained  87%  of  the  variability  of  the  degree  of  ILS 
implementation  for  this  model  and  95%  of  the  variability  of  the  degree  of  ILS  implementation  was  explained  by 
Components  5,  2,  12,  4,  and  6. 

Table  4.  Stepwise  Multiple  Regression  of  Influence  Variables  on  Degree  of  ILS  Implementation 


Variable*'b 

R 

R2 

Adjusted 

R2 

Std.  Error 
of  Estimate 

5.  Integrates  with  Classroom  Instruction 

.748 

.560 

.544 

9.39 

2.  Received  Training  in  Use  of  ILS 

.855 

.732 

.711 

7.47 

12.  Uses  Reinforcement/Motivational  Strategies 

.933 

.871 

.856 

5.28 

4.  Sets  Instructional  Goals  for  ILS 

.967 

.935 

.924 

3.82 

6.  Integrates  with  District  Curriculum 

.974 

.950 

.939 

3.44 

15.  IJnderstands/Uses  ILS  Routines  and  Equipment 

.980 

.961 

.950 

3.11 

8.  Effective  Scheduling  of  ILS 

.984 

.969 

.959 

2.82 

14.  Uses  ILS  Achievement  in  Learner  Evaluation 

.989 

.978 

.969 

2.44 

1 i.  Facilitates  ILS  Instruction 

.993 

.986 

.980 

1.98 

9.  Sets  Clear  Rules  of  ILS  Usec 

.995 

.990 

.984 

1.77 

“Dependent  Variable  is  Degree  of  Implementation 

bStepwise  Method,  Probability-of-F-to-Enter  <=  .050,  Probability-of-F-to-Remove>=  .100 
Probability  of  F-to-enler  = .050  limits  reached 


Implications  for  Educational  Change 

Current  debate  in  the  field  of  educational  change  has  concentrated  on  contrasting  differences  in  reductionist 
and  holistic  orientations  to  scientific  inquiry  (see  Banathy,  1995).  Carr  (1996)  explained  this  debate  by  suggesting 
that  integration  and  separation  represented  two  fundamentally  different  aspects  of  the  sa^e  reality.  Integration  was 
concerned  that  when  a system  was  separated  or  reduced  to  smaller  components  some  of  its  vital  properties  were  lost 
while  separation  was  concerned  that  the  whole  was  too  complex  to  be  studied  or  understood  in  its  entirety. 

The  problem  with  the  approach  of  the  first  decade  of  computer  use  in  schools  was  that  it  was  focused  on 
applying  the  computer  to  the  existing  instructional  delivery  system.  Collins  (1991)  described  American  schools  as  a 
self-sustaining,  interlocking  system  of  institutions  consisting  of  age-graded  schools,  multiple-choice  testing, 
curriculum  and  materials,  teacher  education,  and  lecture  and  recitation  that  naturally  resisted  technology: 

If  you  try  to  introduce  computers  for  students  to  do  their  work,  the  change  will  be  sustained  only  to  the 
degree  that  it  fits  the  prevailing  institutional  structure.  Since  computers  undermine  the  lecture  and  recitation 
methods  of  teaching  and  promote  the  student  as  self-directed  learner,  they  do  not  fit  this  institutional 
structure  and  will  be  squeezed  out  by  it.  (p.  32) 

However,  for  systemic  change  to  occur  the  focus  of  change  must  become  our  view  of  the  role  of  the  teacher 
in  implementing  technology.  Many  researchers  in  the  field  of  educational  change  are  applying  the  principles  of 
holistic  thinking  to  the  creation  of  new  systems  of  human  learning  through  methodologies  that  focus  less  on  end 
goals  and  outcomes  and  more  on  helping  individuals  change  their  perceptions  of  themselves  (Carr,  1996).  The 
methodology  followed  by  this  study — emphasizing  ILS  implementation  factors  and  focusing  on  users  of  ILS 
technology — and  the  subsequent  data  collection  and  findings  support  a systemic  change  process  in  education. 

The  findings  of  this  study  provided  evidence  for  the  proposition  that  not  ail  ILS  use  is  the  same.  Significant 
differences  and  variances  in  both  the  ILS  implementation  concerns  and  behaviors  of  teachers  implementing  an  ILS 
were  noted.  The  level  of  ILS  implementation  was  a function  of  implementation  practices  that  included  integration 
with  classroom  instruction,  training  in  the  use  of  an  ILS,  and  the  use  of  motivational  strategies. 

Furthermore,  using  an  ILS  to  improve  the  teaching/leaming  process  was  more  complex  than  earlier 
understood.  Just  because  a teacher  was  an  effective  implementer  of  an  ILS  was  no  guarantee  that  learners  realized 
higher  achievement  using  the  ILS.  Without  the  necessary  organizational  support  the  expectation  for  instructional 
technology  to  improve  the  teaching  and  learning  process  cannot  be  sustained. 

This  examination  of  the  practices  of  teachers  implementing  an  ILS  can  be  a highly  effective  guide  to  actions 
that  change  agents  and  stakeholders  might  follow  as  they  assume  a role  in  facilitating  change  and  improvement  in  the 
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implementation  of  ILS  technology.  Given  the  context  of  ILS  implementation,  the  findings  of  this  study  substantiated 
the  research  on  change  and  innovation.  Several  lessons  from  the  change  literature  were  revealed  through  this  study 
that  might  assist  'takeholders  or  change  agents  in  promoting  a successful  and  effective  implementation  of  computer- 
delivered  instruction: 

1.  Implementation  of  an  ILS  is  a developmental  process  that  must  be  nurtured  and  sustained.  Implementation  of  an 
ILS  doesn’t  happen  by  itself.  When  left  to  their  own  devices,  teachers  will  implement  an  ILS  to  whatever  level  is 
consistent  with  their  concerns  about  the  ILS  and  structurally  fits  into  their  existing  teaching  patterns  and 
practices. 

2.  Teachers  implement  an  ILS  in  different  ways.  The  specific  operational  components  of  the  ILS  must  be 
communicated  to  teachers  so  they  understand  what  the  program  looks  like  when  it  is  fully  functioning.  Before 
learner  achievement  data  can  accurately  be  examined  to  determine  the  effectiveness  of  the  ILS,  it  must  be 
determined  to  what  degree  the  ILS  has  been  implemented.  A determination  of  the  relative  merit  of  an  ILS  should 
be  based  on  an  examination  of  the  degree  to  which  an  ILS  is  actually  used  in  relation  to  the  intended  use  of  the 
ILS. 

3.  Integrate  ILS  instruction  with  classroom  activities  and  instruction.  For  systemic  change  to  occur  in  the 
teaching/leaming  process,  the  existing  system  has  to  be  fundamentally  replaced  with  an  improved  system. 
Teachers  that  were  the  most  effective  implementers  of  an  ILS  incorporated  ILS  instruction  ink  classroom 
instruction  or  vice  versa. 

4.  Training  should  be  a continuous  process  and  not  a one-time  event.  On-going  training  is  a key  component  to 
sustaining  an  ILS  implementation.  On-going  training  reaffirms  fundamental  practices  that  focus  the  user  on  the 
intended  use  of  the  ILS  and  influences  the  concerns  a user  will  have  about  the  implementation  of  an  ILS. 

5.  ILS  implementation  must  be  teacher-driven.  There  is  a perception  that  an  ILS  runs  itself  (Gleghom,  1993)  and 
by  simply  placing  learners  with  computers  does  not  ensure  that  they  will  grasp  the  underlying  structure  of 
important  ideas  and  concepts.  Resnick  and  Johnson  (1988)  noted  that  learning  that  occurs  in  isolation  is  not 
sustained  in  other  contexts  and  Pea  (1988)  explained  the  necessity  of  teachers  to  assist  learners  in  applying 
principles  in  multiple  situations.  Effective  teaching  practices  accommodate  effective  ILS  implementation 

6.  An  understanding  of  how  the  ILS  is  to  be  used  and  the  expectations  for  learning  must  be  clearly  articulated. 
Jones  (1990)  observed  that  change  is  more  manageable  and  occurs  more  easily  when  school  districts  articulate  a 
common  perception  about  what  the  change  process  entails.  Once  an  ILS  goes  into  a school  building  it  is  beyond 
the  control  of  the  designers  and  developers  and  it  becomes  the  responsibility'  of  the  stakeholders  in  the  change 
process  to  ensure  that  the  ILS  is  properly  implemented  and  that  systemic  change  occurs 

Conclusions 


This  study  focused  on  describing  the  concerns  and  implementation  practices  f teachers  implementing  an 
ILS.  Computer-delivered  instruction  and  integrated  learning  systems  represent  one  stage  in  a long  succession  of 
technologies  and  innovations  embraced  by  education  in  the  last  several  decades.  Although  the  problem  of 
implementing  ILSs  is  relatively  new,  the  issue  of  properly  or  adequately  assimilating  technology  into  the  classroom 
and  promoting  its  effective  use  has  long  been  a matter  of  research  and  debate.  Since  the  implementation  of  ILSs  is  a 
complex  process,  further  studies  that  describe  the  implementation  process  for  ILSs  should.be  conducted. 

Implementation  is  better  understood  when  contextual  phenomena  of  the  implementation  are  examined 
because  the  question  of  the  instructional  effectiveness  of  an  ILS  or  other  forms  of  computer-delivered  instruction 
may  be  as  much  an  implementation  issue  as  a matter  of  instructio,  ‘ design.  Ultimately,  the  potential  of  ILSs  and 
other  forms  of  computer-delivered  instruction  may  be  unfulfilled  wh.  * the  technology  is  ineffectively  or  improperly 
implemented. 
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Computer-Delivered  Instruction  Configuration  Matrix 
Integrated  Learning  System 


Rules  for  Rating: 

(1)  Any  information  in  the  interview  may  be  used  to  rate  any  single  component. 

(2)  Rate  to  the  highest  level  of  use  described  by  the  respondent  for  any  single  component. 


5 

IDEAL  USE 

4 3 

ACCEPTABLE  USE 

2 

MINIMIMAL 

USE 

Component  1 
Instructional 
System  and 
Design  of  ILS 

Describes 

individualized 

prescriptive 

strategies 

Describes 
instructional 
presentation  and 
mastery  of  skills 

Describes 
enrollment  levels 

Describes 
different  content 
areas  only 

Component  2 
Training  in  Use 
oflL.- 

Initial  training, 
continued  training, 
and  program  updates 
are  conducted 

Initial  training  and 
continued  training  is 
conducted 

Initial  training  or 
orientation  is 
conducted 

Training  is  self- 
directed  and 
occurs  on-the- 
job 

Component  3 
On-Going 
Support  System 
and 

Communication 
in  Use  of  ILS 

Formal  grade/ 
department  level 
meetings  to  discuss 
ILS  are  conducted 

Building  level 
meetings  with  vendor 
or  principal  to 
discuss  ILS  are 
conducted 

Technology 
committee  meets 
periodically  to 
discuss  ILS 
instruction 

Informal 
discussions  with 
lab  manager  or 
other  teachers 

Component  4 
Instructional 
Goals  or 
Expectations  for 
Use  of ILS 

Accomplishment  of 
goals  is  celebrated 

Instructional  goals 
for  ILS  are 
accomplished 

A plan  for 
accomplishing 
instructional 
goals  is  stated 

Goals  or 
expectations  for 
ILS  are  stated 

Component  5 

ILS  is  used  as  a tool 

for  regularly 

accomplishing 

classroom 

instructional 

objectives 

Plans  lessons  that 
integrate  ILS 
courseware  with 
classroom  instruction 
in  multiple  subjects 
(Worksheets  may  be 
used) 

Plans  lessons  that 
integrate  ILS 
courseware  with 
classroom 
instruction  in  one 
subject 

(Worksheets  may 
be  used.) 

ILS  courseware 
supplements 
classroom 
instruction 

Integration  of 
ILS  Courseware 
with  Classroom 
Instruction 

Component  6 
Integration  of 
ILS  Courseware 
with  Curriculum 

Sequence  and 
selection  of 
courses/lessons  are 
adjusted  to  align  with 
or  support  district 
curriculum 

ILS  courseware 
supplements  district 
curriculum  in 
multiple  subjects 

ILS  courseware 

supplements 

district 

curriculum  in  one 
subject 

ILS  courseware 
is  correlated  to 
district 
curriculum 
when  possible 

Component  7 
Appropriate 
Selection  of 
Courses, 
Enrollment 
Levels,  and 
Options  of  ILS 

Individualized 
learning  sequences 
are  designed  and 
modified  based  on 
test  scores, 
monitoring  student 
progress,  forecasts  of 
learning  gains 

Learning  sequences 
are  individualized  for 
each  student  based 
on  test  scores  or 
monitoring  of  student 
progress 

Test  scores  or 
prior  ILS 
performance  are 
used  to  enroll 
students  in  same 
courses  at 
different  grade 
levels 

Students  are 
enrolled  in  same 
courses  at  grade 
level 

Component  8 
Effective 

Scheduling  of  ILS 

All  students  are 
scheduled  for  regular 
use  and  makeup 
sessions  are  provided 

All  students  arc 
scheduled  regular  use 

Some  students 
are  scheduled  for 
regular  use 

Some  students 
are  scheduled 
for  occasional, 
remedial  or 
specialized  use 

I 

UNACCEPTABLE 

USE 

Has  no 

understanding  of 
instructional  design 
or  no  understanding 

is  necessary 

Received  no 
training 


No  attention  is 
given  to  on-going 
support 


No  goals  or 
expectations  for 
ILS  instruction  are 
set 


ILS  courseware  is 
not  integrated  with 
classroom 
instruction 


ILS  courseware  is 
not  integrated  with 
district  curriculum 


Students  are 
enrolled  at 
beginning  level  of 
course  or  strand 


Students  are  not 
scheduled  for  either 
occasional  or 
regular  use 
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2 1 

5 4 3 MINIM  IMA  L UNACCEPTABLE 

IDEAL  USE ACCEPTABLE  USE USE USE 

Component  9 Orientation  to  rules  Rules  and  procedures  Rules  and  Some  rules  and  Mo  rules  and 

Clear  Rules  For  and  procedures  is  are  established  and  procedures  are  procedures  are  procedures  arc 

Daily  Procedures  presented  handed  out  to  established  and  established  by  established 

Using  ILS  students  in  printed  posted  in  lab  or  the  teacher 

form  classroom 

Component  10  Amount  of  Stu dents  receive  Students  receive  Students  receive  Students  do  not 

Sufficient  Time  instructional  time  is  more  than  30  minutes  more  than  30  at  least  30  receive  regular 

on  Task  for  Each  determined  by  of  ILS  instruction  per  minutes  of  ILS  minutes  of  ILS  weekly  instruction 

Student  Using  targeted  gain  for  week  and  makeup  instruction  per  instruction  per  using  ILS 

ILS students  sessions  are  provided  week week 

Component  11  Continuously  Continuously  Occasionally  Facilitation  and  Teacher  is  not 

Teacher  facilitates  instruction;  facilitates  instruction  facilitates  intervention  Is  present  or  does  not 

Facilitation  and  provides  intervention  instruction  or  provided  facilitate  ILS 

Intervention  strategies  including  facilitate  when  primarily  by  lab  instruction 

Using  ILS  worksheets,  selected  students  request  manager 

practice,  tutoring,  or  assistance 

small  group 

instruction _ 

Component  12  Recognizes  Recognizes  Recognizes  Explains  Mo  motivational 

Effective  Use  of  individual  and  group  individual  group  reasons  for  strategies  or 

Reinforcement  achievement  through  achievement  through  achievement  using  ILS  and  activities  are  used 

and  Motivational  use  of  individual  and  use  of  individual  through  use  of  encourages 

Strategies  Using  group  motivational  motivational  group  students  to 

ILS  strategies  of  strategies  including  motivational  actively 

motivational  certificates,  wall  strategies  participate  in 

strategies  involving  charts,  or  individual  including  ILS  instruction 

parents  or  competition  contests  or  team 

community  sponsors activities 

Component  13  Reports  are  used  to  Reports  are  used  to  Reports  are  used  Reports  are  used  Reports  are  not  run 

Student  Feedback  provide  information  review  student  for  progress  infrequently  or  or  distributed 

and  Use  of  for  determining  progress  and  modify  review  by  lab  on  a limited 

Reports  classroom  instruction-  student  enrollment  manager,  teacher,  basis 

Generated  by  ILS  or  classroom  or  principal 

activities 

Component  14  Evaluation  or  Students  receive  a Students  receive  ILS  is  optional  ILS  is  not  included 

Instructional  assessment  of  letter  or  numeric  a letter  or  for  inclusion  in  in  the  evaluation  or 

Assessmentof  students  includes  grade  for  ILS  numeric  grade  the  evaluation  assessment  of 

ILS  Courseware  mastery  levels,  lesson  achievement  in  for  ILS  or  assessment  of  students 

completion,  or  multiple  subjects  achievement  in  students 

courseware  content  one  subject 

for  multiple  subjects 

Component  15  Familiar  with  course  Familiar  with  course  Familiar  with  Familiar  with  1 o familiarity  with 

Teacher  content  for  multiple  content  for  multiple  course  content  course  content  course  content, 

Knowledge  and  courses,  can  modify  courses  and  can  and  student  and  student  student  resources, 

Skills  Using  ILS  instructional  levels  or  modify  instructional  resources  for  resources  for  or  management 

Routines  and  other  student  levels  or  other  multiple  courses  one  course  system 

Equipment  enrollment  student  enrollment 

information,  and  can  information 
use  custom  reports  or 
forecasting  reports  to 
make  instructional 
decisions 
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Interview  Guide  for  the  Computer-Delivered  Instruction  Configuration  Matrix 

Is  it  OK  if  we  use  the  term  ILS  to  stand  for  the  integrated  learning  system  including  courseware,  computers,  and  lab? 

1.  Describe  in  your  own  words  the  organization  of  ILS  instruction? 

2.  How  much  formal  training  have  you  had  in  the  use  of  ILS?  V/hat  training  have  you  received  lately? 

3.  What  formal  or  informal  communication  such  as  meetings  or  discussions  ha*  occurred  to  support  you  in  the  use 
of  ILS?  (What  do  you  do  and  who  do  you  tell  if  you  have  a problem?) 

4.  What  goals  or  expectations  do  you  set  for  your  class  for  ILS  instruction?  How  do  you  determine  if  these  goals 
are  accomplished? 

5.  When  planning  for  classroom  instruction,  how  do  you  integrate  or  coordinate  ILS  instruction  into  classroom 
activities? 

6.  Does  ILS  integrate  with  district  or  grade  level  curriculum?  If  so,  in  what  ways  and  with  what  courses? 

7.  How  do  you  determine  the  courses,  level,  and  sequence  of  instruction  students  receive  using  ILS?  Are 
modifications  to  student  enrollment  ever  made?  If  so,  how  do  you  determine  what  modifications  are  made? 

8.  What  students  in  your  class  receive  ILS  instruction?  Are  makeup  classes  provided  for  when  students  miss  ILS 
instruction  or  when  your  class  misses  a scheduled  lab  time? 

9.  Have  rules  or  procedures  been  established  for  students  using  ILS?  If  so,  how  do  students  know  these  rules  or 
procedures? 

10.  How  much  time  do  students  spend  each  day  or  week  using  ILS?  How  do  you  determine  the  amount  of  time 
students  spend  using  ILS? 

1 1.  What  do  you  actually  do  while  the  students  are  using  ILS? 

12.  How  do  you  keep  students  motivated  about  using  ILS?  (Are  there  organized  programs  in  your  classroom  or 
school  to  recognize  student  achievement  using  ILS?) 

13.  How  do  you  use  the  student  reports  generated  by  ILS?  Is  this  information  reported  to  the  principal  or  lab 
manager?  If  so,  how  do'  they  use  the  reports? 

14.  Is  ILS  included  in  your  evaluation  and  assessment  of  students?  If  so,  in  what  ways  do  you  use  ILS  for  evaluation 
and  assessment  of  students? 

15.  What  courses  on  the  ILS  are  you  most  familiar  with?  What  routines  on  ILS  can  yc  - perform  (student  reports, 
custom  reports,  enrollment)? 
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Personalized  Independent  Learning  Systems 

The  1970s  saw  the  birth  of  a new  concept  in  education:  individualized  instruction. 
Technologies  such  as  intellectual  assessment,  programmed  instruction,  and  standardized 
testing  caused  educators  to  rethink  traditional,  group  based  instruction  and  develop  a 
wave  of  individualized  education  systems.  (Armstrong  & Savage  1983,  Bangert,  1983, 
Blackburn,  1976,  Dunn  & Dunn,  1972,  Wilson  & Tosti,  1972).  However,  by  the  mid 
1980'$  individualized  instructional  systems  lost  momentum.  Reports  such  as  the  Bell 
Commission  described  the  United  States  as  “a  nation  at  risk”  due  to  its  failing  educational 
system,  and  educational  reforms  such  as  American  2000  called  not  for  fundamental 
changes  in  the  way  education  is  approached,  but  only  for  higher  educational  standards 
(Seymour  & Seymour,  1992).  As  the  1990s  continue,  little  seems  to  have  improved  for 
American  schools  and  theorists  such  as  Reigeluth  argue  that  American  schools  can 
improve  only  by  committing  to  systemic,  systematic  restructuring  (Reigeluth,  1992). 

Instead  of  offering  piecemeal  solutions  to  educational  change,  educators  need  to  offer 
solutions  that  address  educational  issues  systemically  and  provide  an  overarching  purpose 
or  philosophy  of  education  (Reigeluth,  1992,  Postman,  1993;  1995).  Although  many 
educational  theorists  abandoned  individualized  instruction  in  the  1970s,  individualizing 
instruction  still  hold  promise  as  a an  overarching  philosophical  approach  to  education.  By 
integrating  educational  strategies  that  are  known  to  be  effective,  such  as  reinforcement, 
individual  pacing,  providing  feedback,  mastery  learning,  and  cooperative  learning  into  a 
unified  framework  for  education,  educators  might  be  able  to  develop  functional . 
educational  systems  that  address  students’  needs  (Walberg  et.  al,  1984).  As  emerging 
information  technologies  make  implementing  these  strategies  more  feasible  than  ever 
before,  the  time  to  rethink  individualizing  instruction  has  corne  (Reigeluth,  1992). 

. Theoretical  Framework:  Personalized  Independent  Learning  Systems 
Bichelmeyer  (1997)  provides  a framework  for  rethinking  individualized  instruction.  She 
suggests  that  educators  should  think  of  individualized  learning  as  an  overaching 
philosophy  of  education  with  corresponding  instructional  theories  rather  than  only  an 
instructional  strategy. 


Materials 


Therefore,  in  order  to  be  effective,  instruction  must  be  more  than  tailored  to  the 
individual,  it  must  be  a part  of  a larger  system  designed  to  support  individualized 
learning.  An  effective  individualized  instructional  environment  is  a dynamic  system;  it 
involves  instructors  who  embrace  the  concept  of  individualizing  instruction,  it  provides 
students  guidance  throughout  their  educational  lives,  attending  to  their  individual  needs. 
Bichelmeyer  has  termed  this  type  of  environment  a “Personalized  Independent  Learning 
System”  (PELS). 

In  a Personalized  Independent  Learning  System,  the  learning  experiences  are 
personalized  in  two  ways:  the  content  chosen  reflects  an  individuals’  needs,  and  the 
learning  process  reflects  an  individual’s  needs.  Independence  is  encouraged  in  PILS  by 
focusing  on  producing  autonomous,  self-regulated  learners.  Emphasis  is  placed  on 
developing  learners’  metacognitive  and  critical  thinking  skills  and  ability  to  function  as 
experts  within  their  domain,  rather  than  on  producing  students  who  can  complete  a 
narrow  range  of  instructor  derived  tasks.  In  a PELS,  learning  is  more  important  than 
instruction;  unlike  earlier  individualized  instructional  environments,  a PILS  environment 
is  characterized  by  learning  outcomes  rather  than  the  instructional  materials  involved. 
Last,  a PILS  approaches  education  systemically;  individualized  instructional  approaches 
typically  are  confined  to  particular  materials,  whereas  a PELS  is  an  educational  system, 
designed  with  supports  and  mechanisms  needed  to  support  personalized  learning. 

Networked  computer  technologies  such  as  teleconferencing,  the  World  Wide  Web,  and 
email  can  provide  valuable  support  to  PILS.  Teleconferencing  can  allow  for  groups  of 
learners  with  similar  needs  and  goals  to  meet  irrespective  of  location.  The  World  Wide 
Web  is  a resource  that  can  provide  a wealth  of  information  to  learners  in  the  form  of 
online  textbooks  and  tutorials.  The  web  can  also  support  communication  and  organization 
within  a group  of  learners,  as  they  post  information,  and  consult  course  information  on 
line.  Last,  email  can  provide  students  an  instructors  asynchronous  communication.  Via 
email,  instructors  and  students  can  exchange  ideas  irrespective  to  locale  and  with  greater 
ease  and  convenience.  This  research  project  examines  a series  of  design  courses  that  are 
an  example  of  a PILS  and  evaluates  how  technology  is  used  to  support  a PILS. 

The  design  model  chosen  is  being  used  in  a series  of  courses  delivered  via  distance 
education  to  high  school  students  and  to  a group  of  adult  distance  educators.  The  course 
utilizes  the  World  Wide  Web,  email,  interactive  teleconferencing  to  support 
individualized  learning  over  a distance.  The  researchers  are  using  videotapes  of  the 
interactive  teleconferencing  sessions,  surveys  of  students  and  teachers,  and  analysis  of 
student  reflections  in  order  to  evaluate  the  efficiency,  effectiveness,  and  appeal  of  these 
courses  and  generate  recommendations  toward  the  revision  of  the  PILS  instructional 
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design  theory.  Special  attention  is  also  directed  at  assessing  the  ill-defined  learning 
outcomes  associated  with  teaching  design. 

The  presentation  will  discuss  the  theoretical  framework  of  PILS  and  explore  its 
ramifications.  The  presenters  will  share  their  findings  on  the  strengths  and  weaknesses  of 
the  design  model  and  give  recommendations  on  how  to  use  technology  to  support  and 
create  PELS. 
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